-- Logs begin at Fri 2022-06-24 14:18:12 UTC, end at Fri 2022-06-24 15:42:14 UTC. -- Jun 24 14:59:05.730538 dv-cinder-835384-11-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 24 14:59:05.737996 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 24 14:59:05.740656 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 24 14:59:05 2022] *** Jun 24 14:59:05.740656 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 24 14:59:05.740656 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 24 14:59:05.740656 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: nodename: dv-cinder-835384-11-iscsi Jun 24 14:59:05.740656 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: machine: x86_64 Jun 24 14:59:05.740656 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: clock source: unix Jun 24 14:59:05.740656 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: pcre jit disabled Jun 24 14:59:05.740656 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: detected number of CPU cores: 4 Jun 24 14:59:05.740656 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: current working directory: / Jun 24 14:59:05.740656 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: detected binary path: /usr/bin/uwsgi-core Jun 24 14:59:05.740656 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: your processes number limit is 47680 Jun 24 14:59:05.740656 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: your memory page size is 4096 bytes Jun 24 14:59:05.740656 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: detected max file descriptor number: 2048 Jun 24 14:59:05.740656 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: lock engine: pthread robust mutexes Jun 24 14:59:05.741696 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: thunder lock: enabled Jun 24 14:59:05.741696 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 24 14:59:05.741696 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 24 14:59:05.784108 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: Python main interpreter initialized at 0x562a125719c0 Jun 24 14:59:05.784108 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: python threads support enabled Jun 24 14:59:05.784108 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: your server socket listen backlog is limited to 100 connections Jun 24 14:59:05.784108 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: your mercy for graceful operations on workers is 90 seconds Jun 24 14:59:05.784456 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: mapped 403077 bytes (393 KB) for 2 cores Jun 24 14:59:05.784536 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: *** Operational MODE: preforking *** Jun 24 14:59:05.784785 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: *** uWSGI is running in multiple interpreter mode *** Jun 24 14:59:05.784785 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: spawned uWSGI master process (pid: 80512) Jun 24 14:59:05.784980 dv-cinder-835384-11-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 24 14:59:05.786137 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: spawned uWSGI worker 1 (pid: 80522, cores: 1) Jun 24 14:59:05.786137 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: spawned uWSGI worker 2 (pid: 80523, cores: 1) Jun 24 14:59:05.786137 dv-cinder-835384-11-iscsi devstack@keystone.service[80512]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 24 14:59:11.252773 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 14:59:11.252773 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: from cryptography.utils import int_from_bytes Jun 24 14:59:11.253567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 14:59:11.253567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: from cryptography.utils import int_from_bytes Jun 24 14:59:11.254068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 14:59:11.254068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: from cryptography.utils import int_from_bytes Jun 24 14:59:11.254527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 14:59:11.254527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: from cryptography.utils import int_from_bytes Jun 24 14:59:11.301948 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 14:59:11.302132 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 14:59:11.302323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] command line args: [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 14:59:11.302590 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 14:59:11.302767 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ================================================================================ {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 14:59:11.303018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] admin_token = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.303403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] config_dir = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.303481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 14:59:11.303606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.303709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 14:59:11.303791 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] config_source = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.303868 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] command line args: [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 14:59:11.303940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] debug = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.304124 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.304380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 14:59:11.304380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ================================================================================ {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 14:59:11.304502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.304566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] admin_token = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.304694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.305021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] config_dir = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.305144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.305249 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.305390 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.305495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] config_source = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.305609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] debug = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.305712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.305816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.305936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_config_append = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.305936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.306120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.306220 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_dir = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.306333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.306437 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_file = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.306538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.306640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.306726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.306846 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.306932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.307009 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.307131 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_config_append = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.307222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.307353 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.307430 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.307556 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_dir = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.307633 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.307757 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_file = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.307839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.307911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.308038 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.308120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.308198 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.308319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.308401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.308479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.308605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.308685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.308786 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.308866 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.309035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.309123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.309285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.309366 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.309524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.309604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.309760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.309996 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.310084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.67/identity {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.310194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] publish_errors = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.310314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.310427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.310514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.310655 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.310733 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.310832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.310958 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.311089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.311315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.311479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.67/identity {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.311566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] standard_threads = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.311696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] publish_errors = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.311773 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.311887 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.311988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.312121 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.312208 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.312287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.312364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] use_journal = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.312485 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] use_json = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.312568 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.312646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] use_stderr = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.312758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.312843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] use_syslog = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.312993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] standard_threads = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.313072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.313228 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.313307 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.313472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.313552 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.313650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.313733 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.313831 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] use_journal = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.313954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.314111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] use_json = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.314192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] use_stderr = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.314263 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.314387 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] use_syslog = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.314470 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.314547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:59:11.314650 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.314799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.314879 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.315041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.315120 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.315328 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.315420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.315572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.315652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.315811 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.315888 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.316050 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.316137 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.316280 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.316357 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.316523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.316608 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.316767 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.316853 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.317000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.317077 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.317249 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.317334 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.317485 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.317566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.317723 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.317806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.317954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.318034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.318201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.318279 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.318450 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.318529 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.318690 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.319310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.319472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.319672 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.319752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.319940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.320020 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.320191 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.320347 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.320422 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.320493 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.320676 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.320755 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.320928 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.321016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.321156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.321240 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.321384 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.321468 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.321627 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.321714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.321858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.321935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.322094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.322171 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.322328 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.322416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.322555 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.322638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.322787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.322865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.323017 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.323099 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.323273 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.323346 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.323502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.323569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.323730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.323814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.323960 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.324028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.324192 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.324291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.324436 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.324528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.324664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.324743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.324896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.324975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.325125 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.325209 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.325354 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.325436 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.325582 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.325659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.325816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.325903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.326041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.326135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.326276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.326356 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.326507 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.326586 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.326737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.326816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.326967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.327051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.327223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.327304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.327442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.327522 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.327688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.327767 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.327918 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.328021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.328152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.328233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.328380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.328471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.328616 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.328700 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.328853 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.328939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.329078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.329156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.329306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.329383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.329537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.329616 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.329777 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.329855 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.330016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.330101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.330250 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.330332 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.330495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.330574 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.330731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.330812 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.330961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.331040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.331210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.331338 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.331449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.331527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.331682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.331758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.331904 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.331984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.332144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.332223 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.332378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.332465 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.332611 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.332691 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.332845 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.332925 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.333078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.333170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.333310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.333395 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.333548 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.333625 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.333786 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.333874 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.334019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.334099 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.334253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.334336 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.335061 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.335230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.335465 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.335555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.335706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.335807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.335941 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.336041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.336171 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.336274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.336400 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.336519 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.336630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.336761 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.336864 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.337003 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.337083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.337246 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.337326 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.337485 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.337566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.337724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.337804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.337965 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.338044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.338199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.338281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.338432 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.338512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.338665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.338751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.338902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.338983 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.external = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.339129 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.339258 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.339397 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.339474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.339635 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.339717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.339878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.password = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.339957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.340108 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.token = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.340187 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.340344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.340424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.340579 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.340670 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.340822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.340903 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.341050 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.341131 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.external = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.341282 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.341362 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.341525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.341603 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.341753 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.341833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.341985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.342067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.password = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.342228 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.342308 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.token = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.342473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.342559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.342748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.342835 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.342944 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.343024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.343191 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.343285 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.343431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.343511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.343683 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.343762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.343917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.343997 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.344155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.344236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.344403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.344483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.344651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.344737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.344899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.344979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.345138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.345217 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.345381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.345468 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.345614 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.345692 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.345856 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.345934 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.346092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.346251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.346329 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.346401 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.346574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.346655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.346802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.346890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.347046 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.347133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.347302 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.347385 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.347556 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.347652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.347854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.347944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.348043 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.348123 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.348276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.348369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.348520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.348601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.348754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.348835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.348984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.349070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.349228 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.349309 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.349462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.349540 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.349693 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.349779 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.349939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.350048 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.350707 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.350789 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.350944 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.351165 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.351278 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.351354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.351449 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.351530 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.351676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.351766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.351904 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.351985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.352132 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.352212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.352372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.352461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.352616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.352703 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.352852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.352931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.353095 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.353183 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.353335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.353414 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.353585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.353664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.353820 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.353911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.354049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.354128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.354273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.354351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.354507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.354594 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.354733 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.354813 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.354965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.355045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.355223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.355311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.355462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.355552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.355707 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.355787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.355933 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.356014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.356175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.356254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.356417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.356497 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.356650 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.356731 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.356881 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.356965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.357121 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.357201 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.357347 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.357426 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.357581 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.357660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.357813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.357893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.358049 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.358138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.358281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.358361 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.358517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.358596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.358749 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.358831 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.358983 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.359062 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.359250 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.359331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.359487 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.359569 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.359714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.359792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.359948 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.360026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.360189 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.360373 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.360453 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.360656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.360783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.360896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.361136 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.361216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.361387 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.361615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.361701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.361857 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.362056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.362143 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.362331 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.362491 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.362570 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.362801 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.362939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.363021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.363286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.363424 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.363524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.363767 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.363857 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.364028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.364260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.364339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.364510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.364759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.364846 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.365008 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.365216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.365295 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.365463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.365609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.366434 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.366634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.366711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.366901 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.366982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.367153 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.367287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.367423 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.367536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.367664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.367789 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.367920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.368039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.368174 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.368289 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.368426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.368540 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.368676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.368764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.368924 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.369004 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.369157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.369237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.369406 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.369494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.369646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.369726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.369890 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.369969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.370130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.370218 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.370368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.370448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.370615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.370697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.370859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.370946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.371095 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.371193 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.371365 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.371444 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.371593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.371669 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.371832 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.371910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.372077 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.372155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.372311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.372393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.372547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.372626 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.372780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.372857 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.373016 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.373096 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.373246 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.373326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.373482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.373561 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.373721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.373803 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.373969 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.374049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.374220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.374298 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.374456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.374535 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.374700 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.374778 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.374950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.375028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.375197 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.375277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.375448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.375526 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.375683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.375761 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] role.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.375922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.376004 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] role.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.376153 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.376231 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] role.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.376395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] role.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.376473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.376623 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.376704 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.376871 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.376951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.377117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.377195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.377358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.377438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.377602 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.377681 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.377846 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.377925 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.378089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.378170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.378322 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.378400 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.378564 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.378649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.378795 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.378875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.379039 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.379119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.379305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.379380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.379550 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.379729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.379808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.379970 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.380179 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.380267 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.380417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.380497 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.380671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.380750 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.380918 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.381005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.381151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.381229 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.381421 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.381500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.381654 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.381905 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.381985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.382155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.382235 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.382392 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.382470 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.382621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.382700 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.382850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.382929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.383080 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.383159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.383352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.383434 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.383604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.383684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.383848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.383927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.384082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.384163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.384315 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.384392 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.384540 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.384618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.384771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.384855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.385004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.385083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.385243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.385322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.385473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.385553 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.385724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.385803 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.385963 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.386040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.386191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.386271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.386431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.386511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.386668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.386754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.386905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.386984 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.387135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.387241 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.387391 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.387471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.387630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.387711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.387875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.387954 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.388105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.388183 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.388332 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.388411 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.388571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.388651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.388862 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.388941 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.connection = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.389111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.389207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.connection = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.389361 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.389449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.389598 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.389678 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.389831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.389910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.390067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.390147 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.390298 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.390382 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.390532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.390611 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.390757 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.390834 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.390983 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.391063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.391244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.391337 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.391480 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.391561 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.391721 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.391799 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.391950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.392029 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.392195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.392273 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.392429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.392498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.392670 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.392747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.392893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.392973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.393128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.393207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.393280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 14:59:11.393350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:59:11.393513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 14:59:11.424105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.424324 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.424551 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.424636 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.424787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.425019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.433850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.435341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.461729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.461953 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.462850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.463089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.604556 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.604737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.654113 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.654348 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.655062 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.655329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.718292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.718511 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.718622 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.718734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.718820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.718952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.719044 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.719171 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.719275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.719406 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.719488 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.719622 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.744506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.744653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.744737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.744812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.744952 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.745036 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.745170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.745253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.745383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.745466 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.745594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.745685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.745803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.745886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.756934 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.757256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.769598 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.769832 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.769916 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.770059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.770142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.770279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.770372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.770496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.770579 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.770707 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.770787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.770918 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.771003 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.771128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:59:11.789539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.789746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 14:59:11.789840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.790034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 14:59:11.790122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.790273 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 14:59:11.790357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.790546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.790640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 14:59:11.790809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.790892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.791067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.791150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.791345 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.791442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.791599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.802145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.802304 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.802425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.802578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.802695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.802846 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.802977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.803152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.811585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.811732 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.814891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.815007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.815191 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.815316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.815482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.815595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.815763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.815871 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.816045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.816137 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.816307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.816390 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.816579 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.816661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.816851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.816935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.817120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.817204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.817396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.817477 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.817658 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.817742 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.844727 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.844903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.846339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.846510 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.846620 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.846784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.852561 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.852661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.852959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.853045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.853243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.853327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.860634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.860732 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.860874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.860958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.865375 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.865482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.865708 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.865841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.866000 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.866136 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.866279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.866424 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.866537 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.866685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.866798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.866957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.867066 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.867265 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.867366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.867534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.886216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.886317 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.886476 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.886577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.891109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.891270 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.894736 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.894919 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.895127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.895360 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.900702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.901034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.904297 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.904682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.907945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.908241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.908374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.908511 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.908660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.908796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.908930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.909064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.909197 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.909338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.909471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.909730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.929010 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.929286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.929558 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.929689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.930082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.930212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.930441 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.930713 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.937930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.938616 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.939645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.939914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.940369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.940646 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.943159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.943479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.944290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.944506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.944651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.944813 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.944998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.945128 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.945487 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.945776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.958125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.958401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.958677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.958953 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.959092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.959326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.960204 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.960491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.961161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.961457 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.961749 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.962035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.962330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.962625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.979758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.980606 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.983016 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.983928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.986325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.987259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.988073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.988993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.989694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.989969 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.990258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.990711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:11.991040 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.991425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.997554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.998913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:11.999148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:12.000568 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:12.002377 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.002663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.003047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.003349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.003846 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.004138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.004533 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.004821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.012878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:12.013162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.013449 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.014705 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:12.014993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.015312 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.021265 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:12.021541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.021819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.022669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.022942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.023341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.023624 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.023906 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.024178 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:59:12.024307 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.024452 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.024585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.024753 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.024884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.025018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.025172 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.025313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.025444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.025577 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.025708 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.025903 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.026177 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.026451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.026730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.027011 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.027305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.027593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.027857 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:59:12.063620 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x562a125719c0 pid: 80523 (default app) Jun 24 14:59:12.065981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x562a125719c0 pid: 80522 (default app) Jun 24 14:59:12.066968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3b04b-b05f-4dc1-a829-7cc49f74531a None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:12.067288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3b04b-b05f-4dc1-a829-7cc49f74531a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:12.067564 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3b04b-b05f-4dc1-a829-7cc49f74531a None None] PATH_INFO: `/v3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:12.072542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1/1] 10.222.0.67 () {50 vars in 791 bytes} [Fri Jun 24 14:59:12 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:20.538393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793c9aa-80e5-4422-ac6c-c35149ea51bd None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:20.539046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793c9aa-80e5-4422-ac6c-c35149ea51bd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:20.539046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793c9aa-80e5-4422-ac6c-c35149ea51bd None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:20.543710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1/2] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:20.549129 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b72e8-e1d3-48cb-b69c-0f4f10cfa708 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:20.549549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b72e8-e1d3-48cb-b69c-0f4f10cfa708 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:20.549690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b72e8-e1d3-48cb-b69c-0f4f10cfa708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:20.619159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: INFO dbcounter [None req-459b72e8-e1d3-48cb-b69c-0f4f10cfa708 None None] Registered counter for database keystone Jun 24 14:59:20.640340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG oslo_db.sqlalchemy.engines [None req-459b72e8-e1d3-48cb-b69c-0f4f10cfa708 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80523) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 14:59:20.646436 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writer thread running {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 14:59:20.821248 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG passlib.handlers.bcrypt [None req-459b72e8-e1d3-48cb-b69c-0f4f10cfa708 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80523) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 14:59:20.821656 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG passlib.handlers.bcrypt [None req-459b72e8-e1d3-48cb-b69c-0f4f10cfa708 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80523) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 14:59:20.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-459b72e8-e1d3-48cb-b69c-0f4f10cfa708 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:20.878947 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-459b72e8-e1d3-48cb-b69c-0f4f10cfa708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:20.998312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 2/3] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 24 14:59:21.133026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1793c9aa-80e5-4422-ac6c-c35149ea51bd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:21.202612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: INFO dbcounter [None req-1793c9aa-80e5-4422-ac6c-c35149ea51bd None None] Registered counter for database keystone Jun 24 14:59:21.221535 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG oslo_db.sqlalchemy.engines [None req-1793c9aa-80e5-4422-ac6c-c35149ea51bd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80522) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 14:59:21.228216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writer thread running {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 14:59:21.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1793c9aa-80e5-4422-ac6c-c35149ea51bd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:21.348873 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d101f-5d10-4e93-92af-97bc4e480cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:21.349525 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d101f-5d10-4e93-92af-97bc4e480cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:21.349809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d101f-5d10-4e93-92af-97bc4e480cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:21.350083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d101f-5d10-4e93-92af-97bc4e480cb4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:21.358613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d101f-5d10-4e93-92af-97bc4e480cb4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:21.409583 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b35d101f-5d10-4e93-92af-97bc4e480cb4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 14:59:21.409919 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d101f-5d10-4e93-92af-97bc4e480cb4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:21.419943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 2/4] 10.222.0.67 () {58 vars in 1324 bytes} [Fri Jun 24 14:59:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 14:59:21.425781 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b72e8-e1d3-48cb-b69c-0f4f10cfa708 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:21.440476 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b72e8-e1d3-48cb-b69c-0f4f10cfa708 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:21.444894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6e299-7ea1-4ce7-9089-03d549f223cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:21.446188 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6e299-7ea1-4ce7-9089-03d549f223cf admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:21.446479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6e299-7ea1-4ce7-9089-03d549f223cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:21.446762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6e299-7ea1-4ce7-9089-03d549f223cf admin admin] PATH_INFO: `/v3/projects/4e36bcd54fa5438c93e449d16bda5ca3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:21.448300 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6e299-7ea1-4ce7-9089-03d549f223cf admin admin] RBAC: Authorizing `identity:get_project(project_id=4e36bcd54fa5438c93e449d16bda5ca3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:21.510572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: WARNING keystone.common.rbac_enforcer.enforcer [None req-acb6e299-7ea1-4ce7-9089-03d549f223cf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 14:59:21.510914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6e299-7ea1-4ce7-9089-03d549f223cf admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:21.515522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 3/5] 10.222.0.67 () {56 vars in 1207 bytes} [Fri Jun 24 14:59:21 2022] GET /identity/v3/projects/4e36bcd54fa5438c93e449d16bda5ca3 => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:23.496439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec5cb5-0b99-468f-a438-59b799c97f04 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:23.496975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec5cb5-0b99-468f-a438-59b799c97f04 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:23.496975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec5cb5-0b99-468f-a438-59b799c97f04 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:23.501074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 3/6] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:59:23.506557 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3166b-cca6-49bc-9609-a38990e40366 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:23.506930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3166b-cca6-49bc-9609-a38990e40366 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:23.507121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3166b-cca6-49bc-9609-a38990e40366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:23.526128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2cc3166b-cca6-49bc-9609-a38990e40366 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:23.527788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2cc3166b-cca6-49bc-9609-a38990e40366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:23.535481 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 4/7] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 24 14:59:23.558460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec5cb5-0b99-468f-a438-59b799c97f04 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:23.570242 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec5cb5-0b99-468f-a438-59b799c97f04 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:23.574401 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103a267-7c15-40c5-a1c7-d831f33e2894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:23.575024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103a267-7c15-40c5-a1c7-d831f33e2894 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:23.575342 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103a267-7c15-40c5-a1c7-d831f33e2894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:23.575619 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103a267-7c15-40c5-a1c7-d831f33e2894 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:23.583147 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103a267-7c15-40c5-a1c7-d831f33e2894 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:23.584933 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103a267-7c15-40c5-a1c7-d831f33e2894 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:23.594044 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 4/8] 10.222.0.67 () {58 vars in 1324 bytes} [Fri Jun 24 14:59:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 14:59:23.599326 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3166b-cca6-49bc-9609-a38990e40366 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:23.608968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3166b-cca6-49bc-9609-a38990e40366 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:23.613396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f5efd-bdad-4669-824f-548a0469392c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:23.614256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f5efd-bdad-4669-824f-548a0469392c admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:23.614541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f5efd-bdad-4669-824f-548a0469392c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:23.614820 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f5efd-bdad-4669-824f-548a0469392c admin admin] PATH_INFO: `/v3/users/4f372b5546df46a8836cf73269458aff` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:23.616232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f5efd-bdad-4669-824f-548a0469392c admin admin] RBAC: Authorizing `identity:get_user(user_id=4f372b5546df46a8836cf73269458aff)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:23.618064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f5efd-bdad-4669-824f-548a0469392c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:23.619644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 5/9] 10.222.0.67 () {56 vars in 1198 bytes} [Fri Jun 24 14:59:23 2022] GET /identity/v3/users/4f372b5546df46a8836cf73269458aff => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:25.720709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b12d9-1d0b-4ce2-bee0-67bf1e425d41 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:25.721225 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b12d9-1d0b-4ce2-bee0-67bf1e425d41 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:25.721331 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b12d9-1d0b-4ce2-bee0-67bf1e425d41 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:25.722624 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 5/10] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:25.728498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c499a72-9bd2-4bb2-9707-3d0e3e92683c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:25.728915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c499a72-9bd2-4bb2-9707-3d0e3e92683c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:25.729175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c499a72-9bd2-4bb2-9707-3d0e3e92683c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:25.743442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079eac4-a029-4c62-a140-0d4e2a34be58 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:25.743728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079eac4-a029-4c62-a140-0d4e2a34be58 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:25.744006 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079eac4-a029-4c62-a140-0d4e2a34be58 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:25.745313 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 6/11] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:59:25.746514 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7c499a72-9bd2-4bb2-9707-3d0e3e92683c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:25.747769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7c499a72-9bd2-4bb2-9707-3d0e3e92683c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:25.750407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9a6d5-cdc5-4a7d-9f15-7e4fca38e9a4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:25.750695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9a6d5-cdc5-4a7d-9f15-7e4fca38e9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:25.750970 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9a6d5-cdc5-4a7d-9f15-7e4fca38e9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:25.790012 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG passlib.handlers.bcrypt [None req-e1b9a6d5-cdc5-4a7d-9f15-7e4fca38e9a4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80522) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 14:59:25.790435 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG passlib.handlers.bcrypt [None req-e1b9a6d5-cdc5-4a7d-9f15-7e4fca38e9a4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80522) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 14:59:25.827585 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 6/12] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:59:25.836585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e1b9a6d5-cdc5-4a7d-9f15-7e4fca38e9a4 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:25.838161 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e1b9a6d5-cdc5-4a7d-9f15-7e4fca38e9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:25.849610 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c499a72-9bd2-4bb2-9707-3d0e3e92683c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:25.874240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:25.939483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 7/13] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:59:25.954222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c499a72-9bd2-4bb2-9707-3d0e3e92683c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:25.963912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9a6d5-cdc5-4a7d-9f15-7e4fca38e9a4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:25.987136 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07dfffc-c316-47fa-a781-696591d3cce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:25.987759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dfffc-c316-47fa-a781-696591d3cce2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:25.988050 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dfffc-c316-47fa-a781-696591d3cce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:25.988337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dfffc-c316-47fa-a781-696591d3cce2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:26.056092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07dfffc-c316-47fa-a781-696591d3cce2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:26.057567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07dfffc-c316-47fa-a781-696591d3cce2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:26.062608 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9a6d5-cdc5-4a7d-9f15-7e4fca38e9a4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:26.065004 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:26.081381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:26.100502 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4324968-2748-4f1e-92b4-91aaead6a83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:26.102293 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4324968-2748-4f1e-92b4-91aaead6a83b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:26.103504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4324968-2748-4f1e-92b4-91aaead6a83b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:26.103802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4324968-2748-4f1e-92b4-91aaead6a83b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:26.116047 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4324968-2748-4f1e-92b4-91aaead6a83b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b4324968-2748-4f1e-92b4-91aaead6a83b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:26.127967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:26.129075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:26.129969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:26.129969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:26.129969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:26.129969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:26.129969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:26.129969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:26.129969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:26.129969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:26.129969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:26.129969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:26.129969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:26.129969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 14:59:26.129969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 8/14] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 14:59:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:59:26.134961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4324968-2748-4f1e-92b4-91aaead6a83b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:26.162685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 7/15] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 14:59:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 14:59:26.169099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07dfffc-c316-47fa-a781-696591d3cce2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:26.234431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4324968-2748-4f1e-92b4-91aaead6a83b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:26.266998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07dfffc-c316-47fa-a781-696591d3cce2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:26.267666 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af81c0-a39c-4b22-94c4-95a7e167f2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:26.268334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af81c0-a39c-4b22-94c4-95a7e167f2bd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:26.268614 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af81c0-a39c-4b22-94c4-95a7e167f2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:26.268889 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af81c0-a39c-4b22-94c4-95a7e167f2bd None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:26.269643 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af81c0-a39c-4b22-94c4-95a7e167f2bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:26.269948 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af81c0-a39c-4b22-94c4-95a7e167f2bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:26.271436 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af81c0-a39c-4b22-94c4-95a7e167f2bd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:26.283777 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 9/16] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:26.289783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af81c0-a39c-4b22-94c4-95a7e167f2bd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:26.290787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:26.301983 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d13a9d-398f-4654-ac4b-e380f8d0e1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:26.302786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d13a9d-398f-4654-ac4b-e380f8d0e1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:26.303073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d13a9d-398f-4654-ac4b-e380f8d0e1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:26.303382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d13a9d-398f-4654-ac4b-e380f8d0e1f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:26.314330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d13a9d-398f-4654-ac4b-e380f8d0e1f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:26.315850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d13a9d-398f-4654-ac4b-e380f8d0e1f7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:26.317024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:26.336409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-54d13a9d-398f-4654-ac4b-e380f8d0e1f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:59:26.336409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:26.336409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:26.336409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:26.336409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:26.336409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:26.336409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:26.336409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:26.336409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:26.336409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:26.336409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:26.336409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:26.336409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:59:26.336409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:59:26.336409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:26.337276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:26.338170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:59:26.338170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:59:26.338170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:59:26.338170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 14:59:26.338170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 8/17] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 14:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:59:26.343359 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d13a9d-398f-4654-ac4b-e380f8d0e1f7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:26.391082 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af81c0-a39c-4b22-94c4-95a7e167f2bd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:26.424834 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574da5b9-940b-42e8-b917-2ac74e142ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:26.425813 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574da5b9-940b-42e8-b917-2ac74e142ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:26.426168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574da5b9-940b-42e8-b917-2ac74e142ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:26.426528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574da5b9-940b-42e8-b917-2ac74e142ca7 None admin] PATH_INFO: `/v3/users/4f372b5546df46a8836cf73269458aff` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:26.428091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574da5b9-940b-42e8-b917-2ac74e142ca7 None admin] RBAC: Authorizing `identity:get_user(user_id=4f372b5546df46a8836cf73269458aff)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:26.430351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574da5b9-940b-42e8-b917-2ac74e142ca7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:26.432006 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 10/18] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:26 2022] GET /identity/v3/users/4f372b5546df46a8836cf73269458aff => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:26.437268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574da5b9-940b-42e8-b917-2ac74e142ca7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:26.442882 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d13a9d-398f-4654-ac4b-e380f8d0e1f7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:26.475829 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7def867-1b0f-4971-95f4-302ac55d0bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:26.477104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7def867-1b0f-4971-95f4-302ac55d0bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:26.477387 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7def867-1b0f-4971-95f4-302ac55d0bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:26.477666 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7def867-1b0f-4971-95f4-302ac55d0bd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:26.478394 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7def867-1b0f-4971-95f4-302ac55d0bd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:26.478701 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7def867-1b0f-4971-95f4-302ac55d0bd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:26.480113 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7def867-1b0f-4971-95f4-302ac55d0bd3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:26.497540 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 9/19] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 14:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:26.537572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:26.545368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574da5b9-940b-42e8-b917-2ac74e142ca7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:26.584194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530daa72-54a9-479c-8644-fdb18b8013cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:26.585008 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530daa72-54a9-479c-8644-fdb18b8013cf None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:26.585298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530daa72-54a9-479c-8644-fdb18b8013cf None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:26.585579 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530daa72-54a9-479c-8644-fdb18b8013cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:26.587129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530daa72-54a9-479c-8644-fdb18b8013cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:26.588639 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530daa72-54a9-479c-8644-fdb18b8013cf None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:26.590484 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 11/20] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 14:59:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:26.596173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7def867-1b0f-4971-95f4-302ac55d0bd3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:26.633903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:26.702512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7def867-1b0f-4971-95f4-302ac55d0bd3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:26.737397 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420b441-a985-4266-8355-7dcb4c1de09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:26.738149 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420b441-a985-4266-8355-7dcb4c1de09f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:26.738326 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420b441-a985-4266-8355-7dcb4c1de09f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:26.738605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420b441-a985-4266-8355-7dcb4c1de09f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:26.739465 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420b441-a985-4266-8355-7dcb4c1de09f None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.domain.id=default, user.id=4f372b5546df46a8836cf73269458aff) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:26.739775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420b441-a985-4266-8355-7dcb4c1de09f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:26.741223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420b441-a985-4266-8355-7dcb4c1de09f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:26.749159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 10/21] 10.222.0.67 () {56 vars in 1322 bytes} [Fri Jun 24 14:59:26 2022] GET /identity/v3/role_assignments?user.id=4f372b5546df46a8836cf73269458aff&scope.domain.id=default&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:28.863245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b206a2d-5d4f-489c-9d63-eb02c540e9bc None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:28.864280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b206a2d-5d4f-489c-9d63-eb02c540e9bc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:28.864764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b206a2d-5d4f-489c-9d63-eb02c540e9bc None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:28.866366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 12/22] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:28.872319 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24278b0-fde3-4391-a627-0595b5536cce None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:28.873151 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24278b0-fde3-4391-a627-0595b5536cce None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:28.873648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24278b0-fde3-4391-a627-0595b5536cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:28.904153 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b24278b0-fde3-4391-a627-0595b5536cce None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:28.905680 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b24278b0-fde3-4391-a627-0595b5536cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:28.990274 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 11/23] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:59:29.030733 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b206a2d-5d4f-489c-9d63-eb02c540e9bc None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:29.171839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b206a2d-5d4f-489c-9d63-eb02c540e9bc None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:29.217468 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e00cad-7cdb-493a-9de0-17d47c88d9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:29.218261 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e00cad-7cdb-493a-9de0-17d47c88d9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:29.218539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e00cad-7cdb-493a-9de0-17d47c88d9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:29.218811 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e00cad-7cdb-493a-9de0-17d47c88d9f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:29.222823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0982f34-22ae-4f45-abe0-0c07b0301c66 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:29.223392 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0982f34-22ae-4f45-abe0-0c07b0301c66 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:29.223889 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0982f34-22ae-4f45-abe0-0c07b0301c66 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:29.225460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 12/24] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:29.231033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddcb61-14b7-4cd0-a47c-4d59684a62f3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:29.231824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddcb61-14b7-4cd0-a47c-4d59684a62f3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:29.232170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddcb61-14b7-4cd0-a47c-4d59684a62f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:29.238138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e00cad-7cdb-493a-9de0-17d47c88d9f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:29.239650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e00cad-7cdb-493a-9de0-17d47c88d9f4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:29.259698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f4e00cad-7cdb-493a-9de0-17d47c88d9f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:59:29.259698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:29.259698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:29.259698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:29.259698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:29.259698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:29.259698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:29.259698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:29.259698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:29.259698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:29.259698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:29.259698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:29.259698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:59:29.259698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:59:29.259698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:29.260588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:29.261408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:59:29.261408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:59:29.261408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:59:29.261408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 14:59:29.261408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 13/25] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 14:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:29.272075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e00cad-7cdb-493a-9de0-17d47c88d9f4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:29.272543 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f2ddcb61-14b7-4cd0-a47c-4d59684a62f3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:29.274597 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f2ddcb61-14b7-4cd0-a47c-4d59684a62f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:29.314238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:29.321290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:29.372893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 13/26] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:59:29.374124 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c4f74-5cc3-4e80-bb90-250760eb0a64 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:29.374350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c4f74-5cc3-4e80-bb90-250760eb0a64 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:29.374622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c4f74-5cc3-4e80-bb90-250760eb0a64 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:29.375913 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 14/27] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:29.394919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e649858-413c-4d6e-a13d-dc3c154b4e42 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:29.395242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e649858-413c-4d6e-a13d-dc3c154b4e42 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:29.395508 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e649858-413c-4d6e-a13d-dc3c154b4e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:29.430682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4e649858-413c-4d6e-a13d-dc3c154b4e42 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:29.431833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4e649858-413c-4d6e-a13d-dc3c154b4e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:29.486406 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e00cad-7cdb-493a-9de0-17d47c88d9f4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:29.534474 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d7442-da1e-4a74-afc9-78728da3de84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:29.535171 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d7442-da1e-4a74-afc9-78728da3de84 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:29.535482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d7442-da1e-4a74-afc9-78728da3de84 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:29.535762 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d7442-da1e-4a74-afc9-78728da3de84 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:29.536503 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d7442-da1e-4a74-afc9-78728da3de84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:29.536814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d7442-da1e-4a74-afc9-78728da3de84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:29.540918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d7442-da1e-4a74-afc9-78728da3de84 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:29.579087 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 15/28] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:59:29.579919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e649858-413c-4d6e-a13d-dc3c154b4e42 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:29.589388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 14/29] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 14:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:29.590732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14831935-bf56-4640-9ef8-95a86f0719d5 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:29.591024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14831935-bf56-4640-9ef8-95a86f0719d5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:29.591374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14831935-bf56-4640-9ef8-95a86f0719d5 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:29.592601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 15/30] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:29.598177 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14831935-bf56-4640-9ef8-95a86f0719d5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:29.685456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:29.691962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:29.701482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e649858-413c-4d6e-a13d-dc3c154b4e42 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:29.721177 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14831935-bf56-4640-9ef8-95a86f0719d5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:29.743989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ab2b2-d1c1-4bd0-86c6-7e76913b6301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:29.744669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ab2b2-d1c1-4bd0-86c6-7e76913b6301 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:29.744955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ab2b2-d1c1-4bd0-86c6-7e76913b6301 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:29.745239 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ab2b2-d1c1-4bd0-86c6-7e76913b6301 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:29.745990 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ab2b2-d1c1-4bd0-86c6-7e76913b6301 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:29.748820 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ab2b2-d1c1-4bd0-86c6-7e76913b6301 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:29.765194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6d95f-9d06-4bdc-8a16-b34f38fc1030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:29.767229 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6d95f-9d06-4bdc-8a16-b34f38fc1030 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:29.767229 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6d95f-9d06-4bdc-8a16-b34f38fc1030 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:29.767229 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6d95f-9d06-4bdc-8a16-b34f38fc1030 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:29.767893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6d95f-9d06-4bdc-8a16-b34f38fc1030 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:29.769456 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6d95f-9d06-4bdc-8a16-b34f38fc1030 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:29.951535 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 16/31] 10.222.0.67 () {60 vars in 1131 bytes} [Fri Jun 24 14:59:29 2022] POST /identity/v3/roles => generated 219 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:59:29.953157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ab2b2-d1c1-4bd0-86c6-7e76913b6301 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:30.001967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 16/32] 10.222.0.67 () {60 vars in 1141 bytes} [Fri Jun 24 14:59:29 2022] POST /identity/v3/projects => generated 301 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:59:30.003542 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05139b98-08a7-4968-9ead-d46f3b2f3cd3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:30.003844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05139b98-08a7-4968-9ead-d46f3b2f3cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:30.004130 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05139b98-08a7-4968-9ead-d46f3b2f3cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:30.052442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-05139b98-08a7-4968-9ead-d46f3b2f3cd3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:30.054172 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-05139b98-08a7-4968-9ead-d46f3b2f3cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:30.084637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ab2b2-d1c1-4bd0-86c6-7e76913b6301 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:30.129738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49d2ea-8574-4323-9b7d-a52d4f26d3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:30.130919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49d2ea-8574-4323-9b7d-a52d4f26d3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:30.131570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49d2ea-8574-4323-9b7d-a52d4f26d3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:30.132058 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49d2ea-8574-4323-9b7d-a52d4f26d3d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:30.148960 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49d2ea-8574-4323-9b7d-a52d4f26d3d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-7c49d2ea-8574-4323-9b7d-a52d4f26d3d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:30.159372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:30.160316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:30.161082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:30.161082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:30.161082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:30.161082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:30.161082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:30.161082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:30.161082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:30.161082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:30.161082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:30.161082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:30.161082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:30.161082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 14:59:30.163546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 17/33] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 14:59:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:30.172401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49d2ea-8574-4323-9b7d-a52d4f26d3d4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:30.209427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:30.269895 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 17/34] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:30.295201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05139b98-08a7-4968-9ead-d46f3b2f3cd3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:30.339612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49d2ea-8574-4323-9b7d-a52d4f26d3d4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:30.361337 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:30.417148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05139b98-08a7-4968-9ead-d46f3b2f3cd3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:30.462465 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa3315-cdb0-4a5e-ba7e-c46ab7564698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:30.463080 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa3315-cdb0-4a5e-ba7e-c46ab7564698 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:30.463394 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa3315-cdb0-4a5e-ba7e-c46ab7564698 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:30.463675 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa3315-cdb0-4a5e-ba7e-c46ab7564698 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:30.464450 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa3315-cdb0-4a5e-ba7e-c46ab7564698 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:30.465963 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa3315-cdb0-4a5e-ba7e-c46ab7564698 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:30.489703 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61504e-05ce-492c-8881-dadd87c9292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:30.490391 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61504e-05ce-492c-8881-dadd87c9292d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:30.490680 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61504e-05ce-492c-8881-dadd87c9292d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:30.490960 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61504e-05ce-492c-8881-dadd87c9292d None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:30.492325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61504e-05ce-492c-8881-dadd87c9292d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:30.492651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61504e-05ce-492c-8881-dadd87c9292d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:30.494174 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61504e-05ce-492c-8881-dadd87c9292d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:30.506299 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 18/35] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:30.508314 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be34308-edd3-4029-b990-6a2445301fe3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:30.508558 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be34308-edd3-4029-b990-6a2445301fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:30.508838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be34308-edd3-4029-b990-6a2445301fe3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:30.510236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 19/36] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:30.512926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be34308-edd3-4029-b990-6a2445301fe3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:30.532012 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 18/37] 10.222.0.67 () {60 vars in 1131 bytes} [Fri Jun 24 14:59:30 2022] POST /identity/v3/roles => generated 225 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:59:30.539885 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a6386-c793-424d-8dcd-4a01144eeb85 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:30.539885 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a6386-c793-424d-8dcd-4a01144eeb85 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:30.539885 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a6386-c793-424d-8dcd-4a01144eeb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:30.552620 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3f6a6386-c793-424d-8dcd-4a01144eeb85 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:30.555332 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3f6a6386-c793-424d-8dcd-4a01144eeb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:30.605876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:30.634871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 19/38] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:30.636095 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac017964-3c0a-47ff-8e34-ad13567822eb None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:30.636390 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac017964-3c0a-47ff-8e34-ad13567822eb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:30.636671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac017964-3c0a-47ff-8e34-ad13567822eb None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:30.637984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 20/39] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:30.643768 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b3459-0d87-46b9-8a50-1c95ff1b4993 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:30.644065 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b3459-0d87-46b9-8a50-1c95ff1b4993 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:30.644284 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b3459-0d87-46b9-8a50-1c95ff1b4993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:30.647599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be34308-edd3-4029-b990-6a2445301fe3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:30.663779 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c00b3459-0d87-46b9-8a50-1c95ff1b4993 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:30.665257 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c00b3459-0d87-46b9-8a50-1c95ff1b4993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:30.692308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976a25f-16fb-4302-a097-fba56cf2ce57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:30.693194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976a25f-16fb-4302-a097-fba56cf2ce57 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:30.693485 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976a25f-16fb-4302-a097-fba56cf2ce57 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:30.693769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976a25f-16fb-4302-a097-fba56cf2ce57 None admin] PATH_INFO: `/v3/users/4f372b5546df46a8836cf73269458aff` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:30.695867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976a25f-16fb-4302-a097-fba56cf2ce57 None admin] RBAC: Authorizing `identity:get_user(user_id=4f372b5546df46a8836cf73269458aff)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:30.697438 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976a25f-16fb-4302-a097-fba56cf2ce57 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:30.699104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 20/40] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:30 2022] GET /identity/v3/users/4f372b5546df46a8836cf73269458aff => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:30.699898 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976a25f-16fb-4302-a097-fba56cf2ce57 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:30.717107 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:30.744120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 21/41] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:30.745063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b3459-0d87-46b9-8a50-1c95ff1b4993 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:30.814127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976a25f-16fb-4302-a097-fba56cf2ce57 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:30.846887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df4fbd-8224-4b54-9ae9-591a16a2fb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:30.847720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df4fbd-8224-4b54-9ae9-591a16a2fb74 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:30.848033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df4fbd-8224-4b54-9ae9-591a16a2fb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:30.848140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b3459-0d87-46b9-8a50-1c95ff1b4993 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:30.848306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df4fbd-8224-4b54-9ae9-591a16a2fb74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:30.849576 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df4fbd-8224-4b54-9ae9-591a16a2fb74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:30.851021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df4fbd-8224-4b54-9ae9-591a16a2fb74 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:30.853412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 21/42] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 14:59:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:30.854152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df4fbd-8224-4b54-9ae9-591a16a2fb74 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:30.863900 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:30.882697 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6e2c3-2901-453d-9f93-18a0acfb1cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:30.883533 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6e2c3-2901-453d-9f93-18a0acfb1cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:30.884050 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6e2c3-2901-453d-9f93-18a0acfb1cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:30.884157 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6e2c3-2901-453d-9f93-18a0acfb1cb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:30.885279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6e2c3-2901-453d-9f93-18a0acfb1cb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:30.886724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6e2c3-2901-453d-9f93-18a0acfb1cb8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:30.888381 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 22/43] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 14:59:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:30.889122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6e2c3-2901-453d-9f93-18a0acfb1cb8 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:30.934179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:30.957906 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df4fbd-8224-4b54-9ae9-591a16a2fb74 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:30.988085 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6e2c3-2901-453d-9f93-18a0acfb1cb8 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:30.993622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57830e-c04c-4990-ae59-bb0709bdb802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:30.994287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57830e-c04c-4990-ae59-bb0709bdb802 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:30.994572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57830e-c04c-4990-ae59-bb0709bdb802 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:30.994855 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57830e-c04c-4990-ae59-bb0709bdb802 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:30.995641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57830e-c04c-4990-ae59-bb0709bdb802 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:30.997014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57830e-c04c-4990-ae59-bb0709bdb802 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:31.021347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0477b-74a7-46a4-811b-122494562b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:31.022004 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0477b-74a7-46a4-811b-122494562b83 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:31.022286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0477b-74a7-46a4-811b-122494562b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:31.022561 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0477b-74a7-46a4-811b-122494562b83 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:31.023824 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0477b-74a7-46a4-811b-122494562b83 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:31.025236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0477b-74a7-46a4-811b-122494562b83 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:31.073891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 22/44] 10.222.0.67 () {60 vars in 1131 bytes} [Fri Jun 24 14:59:30 2022] POST /identity/v3/roles => generated 223 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:59:31.074734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57830e-c04c-4990-ae59-bb0709bdb802 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:31.143280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 23/45] 10.222.0.67 () {60 vars in 1141 bytes} [Fri Jun 24 14:59:30 2022] POST /identity/v3/projects => generated 312 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:59:31.152267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:31.193709 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57830e-c04c-4990-ae59-bb0709bdb802 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:31.234843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251ab7f-f2ca-4799-be2c-289a4334be25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:31.235797 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251ab7f-f2ca-4799-be2c-289a4334be25 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:31.236074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251ab7f-f2ca-4799-be2c-289a4334be25 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:31.236355 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251ab7f-f2ca-4799-be2c-289a4334be25 None admin] PATH_INFO: `/v3/domains/default/users/4f372b5546df46a8836cf73269458aff/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:31.238573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251ab7f-f2ca-4799-be2c-289a4334be25 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4f372b5546df46a8836cf73269458aff, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:31.240046 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251ab7f-f2ca-4799-be2c-289a4334be25 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:31.297247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 23/46] 10.222.0.67 () {58 vars in 1361 bytes} [Fri Jun 24 14:59:31 2022] PUT /identity/v3/domains/default/users/4f372b5546df46a8836cf73269458aff/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:59:33.210210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984e89c-39e6-4fca-a66b-c524c6dd6f1b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:33.210835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984e89c-39e6-4fca-a66b-c524c6dd6f1b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:33.210835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984e89c-39e6-4fca-a66b-c524c6dd6f1b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:33.212361 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 24/47] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:33.217436 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30f5d4-1496-4e47-a20b-34b2532f79f0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:33.217832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30f5d4-1496-4e47-a20b-34b2532f79f0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:33.218005 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30f5d4-1496-4e47-a20b-34b2532f79f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:33.235371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ea30f5d4-1496-4e47-a20b-34b2532f79f0 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:33.236890 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ea30f5d4-1496-4e47-a20b-34b2532f79f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:33.304559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 24/48] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:33.327254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984e89c-39e6-4fca-a66b-c524c6dd6f1b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:33.338659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7b292-f74f-4083-86d4-c5ed5ee876e0 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:33.338954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7b292-f74f-4083-86d4-c5ed5ee876e0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:33.339275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7b292-f74f-4083-86d4-c5ed5ee876e0 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:33.340654 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 25/49] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:59:33.345552 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daca861-5c37-400b-a4c4-5871d2999d7d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:33.345788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daca861-5c37-400b-a4c4-5871d2999d7d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:33.346077 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daca861-5c37-400b-a4c4-5871d2999d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:33.362719 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9daca861-5c37-400b-a4c4-5871d2999d7d None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:33.363934 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9daca861-5c37-400b-a4c4-5871d2999d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:33.413397 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:33.423472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:33.430266 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 26/50] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:59:33.430444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984e89c-39e6-4fca-a66b-c524c6dd6f1b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:33.454196 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daca861-5c37-400b-a4c4-5871d2999d7d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:33.464100 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f202e8-2824-42aa-bc4e-e1d6765c4387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:33.464904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f202e8-2824-42aa-bc4e-e1d6765c4387 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:33.465194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f202e8-2824-42aa-bc4e-e1d6765c4387 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:33.465476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f202e8-2824-42aa-bc4e-e1d6765c4387 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:33.466676 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f202e8-2824-42aa-bc4e-e1d6765c4387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:33.468499 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f202e8-2824-42aa-bc4e-e1d6765c4387 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:33.474770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 25/51] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 14:59:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:33.475669 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f202e8-2824-42aa-bc4e-e1d6765c4387 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:33.556695 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daca861-5c37-400b-a4c4-5871d2999d7d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:33.576115 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f202e8-2824-42aa-bc4e-e1d6765c4387 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:33.590876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4947e-2993-41e6-9920-73a82fef758d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:33.591706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4947e-2993-41e6-9920-73a82fef758d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:33.591998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4947e-2993-41e6-9920-73a82fef758d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:33.592277 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4947e-2993-41e6-9920-73a82fef758d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:33.603431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4947e-2993-41e6-9920-73a82fef758d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-81c4947e-2993-41e6-9920-73a82fef758d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:33.610938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:33.611823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:33.612574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:33.612574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:33.612574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:33.612574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:33.612574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:33.612574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:33.612574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:33.612574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:33.612574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:33.612574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:33.612574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:33.612574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 14:59:33.613164 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a5e74-4ac2-4942-9ab4-eb8615caac5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:33.613164 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a5e74-4ac2-4942-9ab4-eb8615caac5f None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:33.613310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 27/52] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 14:59:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:59:33.613373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a5e74-4ac2-4942-9ab4-eb8615caac5f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:33.613373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a5e74-4ac2-4942-9ab4-eb8615caac5f None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:33.613788 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a5e74-4ac2-4942-9ab4-eb8615caac5f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:33.615243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a5e74-4ac2-4942-9ab4-eb8615caac5f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:33.618388 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4947e-2993-41e6-9920-73a82fef758d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:33.663135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:33.703076 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 26/53] 10.222.0.67 () {60 vars in 1140 bytes} [Fri Jun 24 14:59:33 2022] POST /identity/v3/projects => generated 298 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:59:33.717633 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4947e-2993-41e6-9920-73a82fef758d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:33.756111 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b5f55-a3d7-4657-82a6-f2027c744250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:33.756823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b5f55-a3d7-4657-82a6-f2027c744250 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:33.757114 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b5f55-a3d7-4657-82a6-f2027c744250 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:33.757576 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b5f55-a3d7-4657-82a6-f2027c744250 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:33.758447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b5f55-a3d7-4657-82a6-f2027c744250 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:33.758760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b5f55-a3d7-4657-82a6-f2027c744250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:33.760271 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b5f55-a3d7-4657-82a6-f2027c744250 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:33.769572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 28/54] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:33.775741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a5e74-4ac2-4942-9ab4-eb8615caac5f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:33.901617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:33.982534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a5e74-4ac2-4942-9ab4-eb8615caac5f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:34.016286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdc26c-e559-4d2f-9603-7e348a842077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:34.017167 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdc26c-e559-4d2f-9603-7e348a842077 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:34.017455 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdc26c-e559-4d2f-9603-7e348a842077 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:34.017734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdc26c-e559-4d2f-9603-7e348a842077 None admin] PATH_INFO: `/v3/users/4f372b5546df46a8836cf73269458aff` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:34.018928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdc26c-e559-4d2f-9603-7e348a842077 None admin] RBAC: Authorizing `identity:get_user(user_id=4f372b5546df46a8836cf73269458aff)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:34.020433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdc26c-e559-4d2f-9603-7e348a842077 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:34.022370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 27/55] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:33 2022] GET /identity/v3/users/4f372b5546df46a8836cf73269458aff => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:34.028448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b5f55-a3d7-4657-82a6-f2027c744250 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:34.130328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b5f55-a3d7-4657-82a6-f2027c744250 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:34.139014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:34.163075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51122d-e21f-4a59-a36b-f9f0999510c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:34.164475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51122d-e21f-4a59-a36b-f9f0999510c2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:34.164757 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51122d-e21f-4a59-a36b-f9f0999510c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:34.165033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51122d-e21f-4a59-a36b-f9f0999510c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:34.169951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51122d-e21f-4a59-a36b-f9f0999510c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:34.169951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51122d-e21f-4a59-a36b-f9f0999510c2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:34.169951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 29/56] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 14:59:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:34.175245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdc26c-e559-4d2f-9603-7e348a842077 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:34.258441 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:34.274450 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdc26c-e559-4d2f-9603-7e348a842077 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:34.308141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d1b85-6c90-4c8f-851e-597748def505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:34.308799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d1b85-6c90-4c8f-851e-597748def505 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:34.309083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d1b85-6c90-4c8f-851e-597748def505 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:34.309362 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d1b85-6c90-4c8f-851e-597748def505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:34.310193 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d1b85-6c90-4c8f-851e-597748def505 None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.domain.id=default, user.id=4f372b5546df46a8836cf73269458aff) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:34.310509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d1b85-6c90-4c8f-851e-597748def505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:34.311979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d1b85-6c90-4c8f-851e-597748def505 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:34.321266 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 28/57] 10.222.0.67 () {56 vars in 1322 bytes} [Fri Jun 24 14:59:34 2022] GET /identity/v3/role_assignments?user.id=4f372b5546df46a8836cf73269458aff&scope.domain.id=default&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:35.786893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b00c5f-d55a-4602-b3f3-feaedcba46c4 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:35.787471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b00c5f-d55a-4602-b3f3-feaedcba46c4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:35.787471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b00c5f-d55a-4602-b3f3-feaedcba46c4 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:35.789223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 30/58] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:59:35.797858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585f41f-94f1-475b-a0e0-ff05cdab1252 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:35.798302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585f41f-94f1-475b-a0e0-ff05cdab1252 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:35.798427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585f41f-94f1-475b-a0e0-ff05cdab1252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:35.816669 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d585f41f-94f1-475b-a0e0-ff05cdab1252 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:35.817870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d585f41f-94f1-475b-a0e0-ff05cdab1252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:35.885661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 29/59] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:59:35.915035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b00c5f-d55a-4602-b3f3-feaedcba46c4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:36.022916 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b00c5f-d55a-4602-b3f3-feaedcba46c4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:36.056859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090c9b4-495f-4d80-b6c1-0c2481fc976a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:36.057568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090c9b4-495f-4d80-b6c1-0c2481fc976a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:36.057852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090c9b4-495f-4d80-b6c1-0c2481fc976a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:36.058129 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090c9b4-495f-4d80-b6c1-0c2481fc976a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:36.059323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090c9b4-495f-4d80-b6c1-0c2481fc976a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:36.060758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090c9b4-495f-4d80-b6c1-0c2481fc976a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:36.062371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 31/60] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 14:59:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:36.068625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585f41f-94f1-475b-a0e0-ff05cdab1252 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:36.148824 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:36.176042 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585f41f-94f1-475b-a0e0-ff05cdab1252 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:36.212592 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb47fd7-c12d-4e10-8d9e-429fd9612496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:36.213271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb47fd7-c12d-4e10-8d9e-429fd9612496 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:36.213557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb47fd7-c12d-4e10-8d9e-429fd9612496 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:36.213837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb47fd7-c12d-4e10-8d9e-429fd9612496 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:36.214719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb47fd7-c12d-4e10-8d9e-429fd9612496 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:36.216232 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb47fd7-c12d-4e10-8d9e-429fd9612496 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:36.308532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 30/61] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 14:59:36 2022] POST /identity/v3/users => generated 275 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:59:39.362266 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50383ac2-458c-42cc-9ec1-5095965532da None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:39.362754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50383ac2-458c-42cc-9ec1-5095965532da None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:39.362754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50383ac2-458c-42cc-9ec1-5095965532da None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:39.364138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 32/62] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:39.377302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3602c-1d47-4a5b-9be9-34df3c80f432 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:39.377805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3602c-1d47-4a5b-9be9-34df3c80f432 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:39.377879 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3602c-1d47-4a5b-9be9-34df3c80f432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:39.404220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ebc3602c-1d47-4a5b-9be9-34df3c80f432 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:39.405483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ebc3602c-1d47-4a5b-9be9-34df3c80f432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:39.497538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 31/63] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:39.529902 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76456e09-388f-4f5e-b735-0bdf70b23da5 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:39.530520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76456e09-388f-4f5e-b735-0bdf70b23da5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:39.530998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76456e09-388f-4f5e-b735-0bdf70b23da5 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:39.532645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 33/64] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:39.537809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76456e09-388f-4f5e-b735-0bdf70b23da5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:39.537948 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9b219-909d-409f-b742-3c72055eb02e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:39.538256 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9b219-909d-409f-b742-3c72055eb02e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:39.538784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9b219-909d-409f-b742-3c72055eb02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:39.556420 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e3d9b219-909d-409f-b742-3c72055eb02e None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:39.557938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e3d9b219-909d-409f-b742-3c72055eb02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:39.624103 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:39.648937 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:39.661576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 32/65] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:39.701675 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9b219-909d-409f-b742-3c72055eb02e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:39.721413 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76456e09-388f-4f5e-b735-0bdf70b23da5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:39.774789 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecd6c2-5792-418d-8bbf-3d9c8bffc9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:39.775904 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecd6c2-5792-418d-8bbf-3d9c8bffc9ed None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:39.776338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecd6c2-5792-418d-8bbf-3d9c8bffc9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:39.777212 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecd6c2-5792-418d-8bbf-3d9c8bffc9ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:39.791422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ecd6c2-5792-418d-8bbf-3d9c8bffc9ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a7ecd6c2-5792-418d-8bbf-3d9c8bffc9ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:39.801447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:39.802531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:39.803536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:39.803536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:39.803536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:39.803536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:39.803536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:39.803536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:39.803536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:39.803536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:39.803536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:39.803536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:39.803536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:59:39.803536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 14:59:39.806897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 34/66] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:39.808759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab7f3f-2015-42ab-819f-a38af20996b6 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:39.809523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab7f3f-2015-42ab-819f-a38af20996b6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:39.809954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab7f3f-2015-42ab-819f-a38af20996b6 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:39.812315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 35/67] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:39.817474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab7f3f-2015-42ab-819f-a38af20996b6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:39.838949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9b219-909d-409f-b742-3c72055eb02e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:39.876157 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd573164-5579-499b-a761-e022b5b6f796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:39.877312 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd573164-5579-499b-a761-e022b5b6f796 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:39.877751 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd573164-5579-499b-a761-e022b5b6f796 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:39.878167 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd573164-5579-499b-a761-e022b5b6f796 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:39.889299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd573164-5579-499b-a761-e022b5b6f796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-bd573164-5579-499b-a761-e022b5b6f796 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:39.897166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:39.898158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:39.898930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:39.898930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:39.898930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:39.898930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:39.898930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:39.898930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:39.898930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:39.898930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:39.898930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:39.898930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:39.898930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:39.898930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 14:59:39.901370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 33/68] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 14:59:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:59:39.902751 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2fa86-db70-4d48-babc-c2ef4f14b257 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:39.903357 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2fa86-db70-4d48-babc-c2ef4f14b257 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:39.903812 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2fa86-db70-4d48-babc-c2ef4f14b257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:39.921612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-92e2fa86-db70-4d48-babc-c2ef4f14b257 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:39.923100 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-92e2fa86-db70-4d48-babc-c2ef4f14b257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:39.958247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab7f3f-2015-42ab-819f-a38af20996b6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:39.961657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:39.983033 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:40.003396 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 34/69] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:40.004510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2fa86-db70-4d48-babc-c2ef4f14b257 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:40.022330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5f77a-9c37-4a54-89d9-67493c8dfe3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:40.023867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5f77a-9c37-4a54-89d9-67493c8dfe3f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:40.024399 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5f77a-9c37-4a54-89d9-67493c8dfe3f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:40.024870 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5f77a-9c37-4a54-89d9-67493c8dfe3f None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:40.025889 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5f77a-9c37-4a54-89d9-67493c8dfe3f None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:40.026396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5f77a-9c37-4a54-89d9-67493c8dfe3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:40.029203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5f77a-9c37-4a54-89d9-67493c8dfe3f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:40.044673 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 36/70] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 14:59:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:40.045606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5f77a-9c37-4a54-89d9-67493c8dfe3f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:40.133754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2fa86-db70-4d48-babc-c2ef4f14b257 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:40.166478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5f77a-9c37-4a54-89d9-67493c8dfe3f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:40.168144 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e7754-3b47-4ddd-80bc-f9fd32cb1407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:40.168822 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e7754-3b47-4ddd-80bc-f9fd32cb1407 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:40.169108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e7754-3b47-4ddd-80bc-f9fd32cb1407 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:40.169395 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e7754-3b47-4ddd-80bc-f9fd32cb1407 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:40.170140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e7754-3b47-4ddd-80bc-f9fd32cb1407 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:40.170451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e7754-3b47-4ddd-80bc-f9fd32cb1407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:40.171861 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e7754-3b47-4ddd-80bc-f9fd32cb1407 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:40.180371 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 35/71] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:40.182177 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e7754-3b47-4ddd-80bc-f9fd32cb1407 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:40.200685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c43ca-6364-4d9d-b93a-fceb1532809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:40.201573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c43ca-6364-4d9d-b93a-fceb1532809e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:40.201870 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c43ca-6364-4d9d-b93a-fceb1532809e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:40.202155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c43ca-6364-4d9d-b93a-fceb1532809e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:40.212749 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c43ca-6364-4d9d-b93a-fceb1532809e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-897c43ca-6364-4d9d-b93a-fceb1532809e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:40.220477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:40.221381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:40.222161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:40.222161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:40.222161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:40.222161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:40.222161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:40.222161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:40.222161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:40.222161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:40.222161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:40.222161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:40.222161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:59:40.222161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 14:59:40.222161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 37/72] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:40.223083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d709f8-bc9a-4484-abe5-ac9c636e76bc None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:40.223211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d709f8-bc9a-4484-abe5-ac9c636e76bc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:40.223490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d709f8-bc9a-4484-abe5-ac9c636e76bc None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:40.224780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 38/73] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:40.227357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c578d5-c718-45ef-9b94-0d9bff0a3be2 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:40.227649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c578d5-c718-45ef-9b94-0d9bff0a3be2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:40.227935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c578d5-c718-45ef-9b94-0d9bff0a3be2 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:40.229194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 39/74] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:40.231547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c578d5-c718-45ef-9b94-0d9bff0a3be2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:40.234578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:40.259583 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:40.287492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e7754-3b47-4ddd-80bc-f9fd32cb1407 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:40.320585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582126f-f87a-4b20-a5bf-87303f439274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:40.321459 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582126f-f87a-4b20-a5bf-87303f439274 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:40.321746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582126f-f87a-4b20-a5bf-87303f439274 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:40.322026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582126f-f87a-4b20-a5bf-87303f439274 None admin] PATH_INFO: `/v3/users/2bf0569100d34ba7939616d4bcef590a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:40.334985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c578d5-c718-45ef-9b94-0d9bff0a3be2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:40.337350 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582126f-f87a-4b20-a5bf-87303f439274 None admin] RBAC: Authorizing `identity:get_user(user_id=2bf0569100d34ba7939616d4bcef590a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:40.338800 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582126f-f87a-4b20-a5bf-87303f439274 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:40.340500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 36/75] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:40 2022] GET /identity/v3/users/2bf0569100d34ba7939616d4bcef590a => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:40.341262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582126f-f87a-4b20-a5bf-87303f439274 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:40.373415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b053fd4-606a-4f7c-803b-5bb1303e55fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:40.374300 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b053fd4-606a-4f7c-803b-5bb1303e55fe None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:40.374591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b053fd4-606a-4f7c-803b-5bb1303e55fe None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:40.374875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b053fd4-606a-4f7c-803b-5bb1303e55fe None admin] PATH_INFO: `/v3/users/4f372b5546df46a8836cf73269458aff` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:40.376113 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b053fd4-606a-4f7c-803b-5bb1303e55fe None admin] RBAC: Authorizing `identity:get_user(user_id=4f372b5546df46a8836cf73269458aff)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:40.377887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b053fd4-606a-4f7c-803b-5bb1303e55fe None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:40.379563 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 40/76] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:40 2022] GET /identity/v3/users/4f372b5546df46a8836cf73269458aff => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:40.380779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95e679-d99a-49f2-93e9-f5c511c90c02 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:40.381074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95e679-d99a-49f2-93e9-f5c511c90c02 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:40.381356 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95e679-d99a-49f2-93e9-f5c511c90c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:40.399714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-af95e679-d99a-49f2-93e9-f5c511c90c02 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:40.400839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-af95e679-d99a-49f2-93e9-f5c511c90c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:40.444017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582126f-f87a-4b20-a5bf-87303f439274 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:40.447266 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:40.466981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 41/77] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:40.468280 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4ff50-684c-4f80-ab64-3f33cfbb0d4f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:40.468574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4ff50-684c-4f80-ab64-3f33cfbb0d4f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:40.468855 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4ff50-684c-4f80-ab64-3f33cfbb0d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:40.484556 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-76d4ff50-684c-4f80-ab64-3f33cfbb0d4f None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:40.485734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-76d4ff50-684c-4f80-ab64-3f33cfbb0d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:40.491253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dc7ee-d5bc-47e5-a72f-0126a4bf6e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:40.491997 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dc7ee-d5bc-47e5-a72f-0126a4bf6e19 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:40.492290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dc7ee-d5bc-47e5-a72f-0126a4bf6e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:40.492577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dc7ee-d5bc-47e5-a72f-0126a4bf6e19 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:40.493343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987dc7ee-d5bc-47e5-a72f-0126a4bf6e19 None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:40.493657 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987dc7ee-d5bc-47e5-a72f-0126a4bf6e19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:40.495046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987dc7ee-d5bc-47e5-a72f-0126a4bf6e19 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:40.502693 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:40.503552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 37/78] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 14:59:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:40.504188 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dc7ee-d5bc-47e5-a72f-0126a4bf6e19 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:40.553984 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 42/79] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:40.554992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4ff50-684c-4f80-ab64-3f33cfbb0d4f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:40.606877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dc7ee-d5bc-47e5-a72f-0126a4bf6e19 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:40.639477 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b50c5d-14a7-414c-a3e8-b9d76fa263b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:40.640123 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b50c5d-14a7-414c-a3e8-b9d76fa263b2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:40.640407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b50c5d-14a7-414c-a3e8-b9d76fa263b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:40.640685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b50c5d-14a7-414c-a3e8-b9d76fa263b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:40.652570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4ff50-684c-4f80-ab64-3f33cfbb0d4f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:40.674046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:40.685197 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb20e7-581a-4b51-a277-6f8bfa32667e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:40.685849 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb20e7-581a-4b51-a277-6f8bfa32667e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:40.686136 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb20e7-581a-4b51-a277-6f8bfa32667e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:40.686413 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb20e7-581a-4b51-a277-6f8bfa32667e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:40.708103 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b50c5d-14a7-414c-a3e8-b9d76fa263b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:40.709872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b50c5d-14a7-414c-a3e8-b9d76fa263b2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:40.711459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:40.754401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb20e7-581a-4b51-a277-6f8bfa32667e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:40.755902 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb20e7-581a-4b51-a277-6f8bfa32667e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:40.806874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 38/80] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 14:59:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:59:40.807763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b50c5d-14a7-414c-a3e8-b9d76fa263b2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:40.856576 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 43/81] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 14:59:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:59:40.857611 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb20e7-581a-4b51-a277-6f8bfa32667e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:40.875150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:40.908782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b50c5d-14a7-414c-a3e8-b9d76fa263b2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:40.915349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:40.942647 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab89a71-7689-4e99-bdb0-b229fde4f953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:40.943501 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab89a71-7689-4e99-bdb0-b229fde4f953 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:40.943791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab89a71-7689-4e99-bdb0-b229fde4f953 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:40.944074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab89a71-7689-4e99-bdb0-b229fde4f953 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:40.954624 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab89a71-7689-4e99-bdb0-b229fde4f953 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:40.959624 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb20e7-581a-4b51-a277-6f8bfa32667e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-1ab89a71-7689-4e99-bdb0-b229fde4f953 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:40.962833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:40.963748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:40.964509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:40.964509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:40.964509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:40.964509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:40.964509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:40.964509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:40.964509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:40.964509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:40.964509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:40.964509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:40.964509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:59:40.964509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 14:59:40.964509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 39/82] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 14:59:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:40.965223 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab89a71-7689-4e99-bdb0-b229fde4f953 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:40.994075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f4651-d68d-4afa-807a-c7d585d4c116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:40.994957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f4651-d68d-4afa-807a-c7d585d4c116 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:40.995275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f4651-d68d-4afa-807a-c7d585d4c116 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:40.995554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f4651-d68d-4afa-807a-c7d585d4c116 None admin] PATH_INFO: `/v3/users/2bf0569100d34ba7939616d4bcef590a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:40.996752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f4651-d68d-4afa-807a-c7d585d4c116 None admin] RBAC: Authorizing `identity:get_user(user_id=2bf0569100d34ba7939616d4bcef590a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:40.998194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f4651-d68d-4afa-807a-c7d585d4c116 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:40.999863 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 44/83] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:40 2022] GET /identity/v3/users/2bf0569100d34ba7939616d4bcef590a => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:41.000611 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f4651-d68d-4afa-807a-c7d585d4c116 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:41.063827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab89a71-7689-4e99-bdb0-b229fde4f953 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:41.089092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:41.099730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fe823-2fa8-4e07-a2a1-175613ce41a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:41.100597 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fe823-2fa8-4e07-a2a1-175613ce41a4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:41.101555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fe823-2fa8-4e07-a2a1-175613ce41a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:41.101835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fe823-2fa8-4e07-a2a1-175613ce41a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:41.103025 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fe823-2fa8-4e07-a2a1-175613ce41a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:41.103306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f4651-d68d-4afa-807a-c7d585d4c116 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:41.104555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fe823-2fa8-4e07-a2a1-175613ce41a4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:41.106167 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 40/84] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 14:59:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:41.108095 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fe823-2fa8-4e07-a2a1-175613ce41a4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:41.126536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:41.139846 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c8b6a-6ad3-4580-8cad-0b63cfa8065e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:41.140722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c8b6a-6ad3-4580-8cad-0b63cfa8065e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:41.141008 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c8b6a-6ad3-4580-8cad-0b63cfa8065e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:41.141284 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c8b6a-6ad3-4580-8cad-0b63cfa8065e None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:41.142472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c8b6a-6ad3-4580-8cad-0b63cfa8065e None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:41.143948 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c8b6a-6ad3-4580-8cad-0b63cfa8065e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:41.145648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 45/85] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:40 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:41.146403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c8b6a-6ad3-4580-8cad-0b63cfa8065e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:41.211390 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fe823-2fa8-4e07-a2a1-175613ce41a4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:41.244228 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c8b6a-6ad3-4580-8cad-0b63cfa8065e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:41.247117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e2028-6622-4965-be48-3f03e5ebc5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:41.248008 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e2028-6622-4965-be48-3f03e5ebc5ae None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:41.248298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e2028-6622-4965-be48-3f03e5ebc5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:41.248575 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e2028-6622-4965-be48-3f03e5ebc5ae None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:41.249749 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e2028-6622-4965-be48-3f03e5ebc5ae None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:41.251218 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e2028-6622-4965-be48-3f03e5ebc5ae None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:41.252906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 41/86] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:41 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:41.253654 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e2028-6622-4965-be48-3f03e5ebc5ae None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:41.277340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfab65-64a8-4a24-9725-6813696da4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:41.278011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfab65-64a8-4a24-9725-6813696da4db None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:41.278298 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfab65-64a8-4a24-9725-6813696da4db None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:41.278576 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfab65-64a8-4a24-9725-6813696da4db None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:41.279356 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfab65-64a8-4a24-9725-6813696da4db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:41.279674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfab65-64a8-4a24-9725-6813696da4db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:41.281040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfab65-64a8-4a24-9725-6813696da4db None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:41.289415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 46/87] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 14:59:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:41.290173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfab65-64a8-4a24-9725-6813696da4db None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:41.315846 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:41.341220 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:41.353874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e2028-6622-4965-be48-3f03e5ebc5ae None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:41.387399 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0a043-f99b-4ad4-a528-16221c90d046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:41.387573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfab65-64a8-4a24-9725-6813696da4db None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:41.388058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0a043-f99b-4ad4-a528-16221c90d046 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:41.388347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0a043-f99b-4ad4-a528-16221c90d046 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:41.388631 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0a043-f99b-4ad4-a528-16221c90d046 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:41.420321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347b7b0-e193-44f1-8790-9f29fa92148c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:41.420991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347b7b0-e193-44f1-8790-9f29fa92148c None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:41.421277 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347b7b0-e193-44f1-8790-9f29fa92148c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:41.421554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347b7b0-e193-44f1-8790-9f29fa92148c None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:41.422272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347b7b0-e193-44f1-8790-9f29fa92148c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:41.423748 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347b7b0-e193-44f1-8790-9f29fa92148c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:41.454527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0a043-f99b-4ad4-a528-16221c90d046 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:41.456000 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0a043-f99b-4ad4-a528-16221c90d046 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:41.496510 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 47/88] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 14:59:41 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:59:41.504301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347b7b0-e193-44f1-8790-9f29fa92148c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:41.516060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:41.560312 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 42/89] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 14:59:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:59:41.561185 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0a043-f99b-4ad4-a528-16221c90d046 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:41.622519 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347b7b0-e193-44f1-8790-9f29fa92148c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:41.632687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:41.659894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fcabe-0a2b-4659-8251-0d4c809896db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:41.660875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fcabe-0a2b-4659-8251-0d4c809896db None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:41.660875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fcabe-0a2b-4659-8251-0d4c809896db None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:41.661805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fcabe-0a2b-4659-8251-0d4c809896db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:41.662659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fcabe-0a2b-4659-8251-0d4c809896db None admin] RBAC: Adding query filter params (role.id=b6dd71d4043d43e9823ec25176863347, scope.project.id=4d31d318708e4e9caa66106ce370a235, user.id=2bf0569100d34ba7939616d4bcef590a) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:41.662978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fcabe-0a2b-4659-8251-0d4c809896db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:41.665420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fcabe-0a2b-4659-8251-0d4c809896db None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:41.672911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0a043-f99b-4ad4-a528-16221c90d046 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:41.676882 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 48/90] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 14:59:41 2022] GET /identity/v3/role_assignments?user.id=2bf0569100d34ba7939616d4bcef590a&scope.project.id=4d31d318708e4e9caa66106ce370a235&role.id=b6dd71d4043d43e9823ec25176863347 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:41.680388 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fcabe-0a2b-4659-8251-0d4c809896db None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:41.734368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c1712-c041-472c-a72a-554d74712767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:41.734368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c1712-c041-472c-a72a-554d74712767 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:41.734368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c1712-c041-472c-a72a-554d74712767 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:41.734368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c1712-c041-472c-a72a-554d74712767 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:41.734784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c1712-c041-472c-a72a-554d74712767 None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=4d31d318708e4e9caa66106ce370a235, user.id=4f372b5546df46a8836cf73269458aff) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:41.734784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c1712-c041-472c-a72a-554d74712767 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:41.737326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c1712-c041-472c-a72a-554d74712767 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:41.743819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 43/91] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 14:59:41 2022] GET /identity/v3/role_assignments?user.id=4f372b5546df46a8836cf73269458aff&scope.project.id=4d31d318708e4e9caa66106ce370a235&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:41.745065 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c1712-c041-472c-a72a-554d74712767 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:41.789552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:41.829958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fcabe-0a2b-4659-8251-0d4c809896db None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:41.885251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77553179-a6c2-4260-9dc3-ed23390e657e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:41.885459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77553179-a6c2-4260-9dc3-ed23390e657e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:41.885639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77553179-a6c2-4260-9dc3-ed23390e657e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:41.885925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77553179-a6c2-4260-9dc3-ed23390e657e None admin] PATH_INFO: `/v3/users/2bf0569100d34ba7939616d4bcef590a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:41.889410 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77553179-a6c2-4260-9dc3-ed23390e657e None admin] RBAC: Authorizing `identity:get_user(user_id=2bf0569100d34ba7939616d4bcef590a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:41.889565 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77553179-a6c2-4260-9dc3-ed23390e657e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:41.893194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 49/92] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:41 2022] GET /identity/v3/users/2bf0569100d34ba7939616d4bcef590a => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:41.897076 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77553179-a6c2-4260-9dc3-ed23390e657e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:41.940422 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c1712-c041-472c-a72a-554d74712767 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:41.973803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:41.986883 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173680f-394a-4152-aabf-3c7176ddba22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:41.987869 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173680f-394a-4152-aabf-3c7176ddba22 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:41.990736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173680f-394a-4152-aabf-3c7176ddba22 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:41.990736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173680f-394a-4152-aabf-3c7176ddba22 None admin] PATH_INFO: `/v3/projects/05c0e94829d343e78450ed30009cccc0` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:41.990736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173680f-394a-4152-aabf-3c7176ddba22 None admin] RBAC: Authorizing `identity:get_project(project_id=05c0e94829d343e78450ed30009cccc0)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:41.991433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173680f-394a-4152-aabf-3c7176ddba22 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:41.993688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 44/93] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:41 2022] GET /identity/v3/projects/05c0e94829d343e78450ed30009cccc0 => generated 312 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:41.999460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173680f-394a-4152-aabf-3c7176ddba22 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:42.053618 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77553179-a6c2-4260-9dc3-ed23390e657e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:42.159910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be74e3-2bdc-4649-89c1-48911bb91439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:42.160975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be74e3-2bdc-4649-89c1-48911bb91439 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:42.161490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be74e3-2bdc-4649-89c1-48911bb91439 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:42.161980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be74e3-2bdc-4649-89c1-48911bb91439 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:42.207841 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:42.211869 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173680f-394a-4152-aabf-3c7176ddba22 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:42.276875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7eeaf-1544-458e-8603-425544d2fa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:42.277561 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7eeaf-1544-458e-8603-425544d2fa48 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:42.277859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7eeaf-1544-458e-8603-425544d2fa48 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:42.278146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7eeaf-1544-458e-8603-425544d2fa48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:42.279002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7eeaf-1544-458e-8603-425544d2fa48 None admin] RBAC: Adding query filter params (role.id=b6dd71d4043d43e9823ec25176863347, scope.project.id=05c0e94829d343e78450ed30009cccc0, user.id=2bf0569100d34ba7939616d4bcef590a) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:42.279384 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7eeaf-1544-458e-8603-425544d2fa48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:42.280855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7eeaf-1544-458e-8603-425544d2fa48 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:42.296049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 45/94] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 14:59:41 2022] GET /identity/v3/role_assignments?user.id=2bf0569100d34ba7939616d4bcef590a&scope.project.id=05c0e94829d343e78450ed30009cccc0&role.id=b6dd71d4043d43e9823ec25176863347 => generated 259 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:42.323016 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be74e3-2bdc-4649-89c1-48911bb91439 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:42.324902 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be74e3-2bdc-4649-89c1-48911bb91439 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:42.437328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:42.535986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 50/95] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 14:59:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 14:59:42.544300 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7eeaf-1544-458e-8603-425544d2fa48 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:42.773536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7eeaf-1544-458e-8603-425544d2fa48 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:42.826509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89aa33-d8ed-49d6-b2b5-07c0508357b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:42.827410 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89aa33-d8ed-49d6-b2b5-07c0508357b6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:42.827699 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89aa33-d8ed-49d6-b2b5-07c0508357b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:42.827981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89aa33-d8ed-49d6-b2b5-07c0508357b6 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:42.829177 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89aa33-d8ed-49d6-b2b5-07c0508357b6 None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:42.830613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89aa33-d8ed-49d6-b2b5-07c0508357b6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:42.836877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 46/96] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:42 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:42.844517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be74e3-2bdc-4649-89c1-48911bb91439 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:43.060104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be74e3-2bdc-4649-89c1-48911bb91439 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:43.129383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533b8b7-2afa-45d8-9880-f93eaa207d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:43.130412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533b8b7-2afa-45d8-9880-f93eaa207d81 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:43.130922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533b8b7-2afa-45d8-9880-f93eaa207d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:43.131424 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533b8b7-2afa-45d8-9880-f93eaa207d81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:43.133034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533b8b7-2afa-45d8-9880-f93eaa207d81 None admin] RBAC: Adding query filter params (role.id=c077ebdf7f214d7585dfc1570eecf882, scope.project.id=4d31d318708e4e9caa66106ce370a235, user.id=2bf0569100d34ba7939616d4bcef590a) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:43.133552 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533b8b7-2afa-45d8-9880-f93eaa207d81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:43.135238 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533b8b7-2afa-45d8-9880-f93eaa207d81 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:43.141293 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 51/97] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 14:59:42 2022] GET /identity/v3/role_assignments?user.id=2bf0569100d34ba7939616d4bcef590a&scope.project.id=4d31d318708e4e9caa66106ce370a235&role.id=c077ebdf7f214d7585dfc1570eecf882 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:44.661861 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b4040-8e4f-4e1d-a1b5-af5de6627322 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:44.662726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b4040-8e4f-4e1d-a1b5-af5de6627322 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:44.663231 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b4040-8e4f-4e1d-a1b5-af5de6627322 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:44.665350 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 47/98] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:44.670678 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f251f-6e9a-47dd-9ada-27512519fd46 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:44.671544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f251f-6e9a-47dd-9ada-27512519fd46 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:44.672021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f251f-6e9a-47dd-9ada-27512519fd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:44.683350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:44.699904 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1e1f251f-6e9a-47dd-9ada-27512519fd46 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:44.703253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602762d-ee5e-47da-a413-d7ae968d3000 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:44.703253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602762d-ee5e-47da-a413-d7ae968d3000 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:44.703253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602762d-ee5e-47da-a413-d7ae968d3000 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:44.703704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1e1f251f-6e9a-47dd-9ada-27512519fd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:44.703776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 48/99] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:44.707644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff1490-c524-4c09-b536-0249452d6df9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:44.708199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff1490-c524-4c09-b536-0249452d6df9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:44.708672 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff1490-c524-4c09-b536-0249452d6df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:44.733529 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4cff1490-c524-4c09-b536-0249452d6df9 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:44.735040 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4cff1490-c524-4c09-b536-0249452d6df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:44.780039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:44.818266 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 52/100] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:44.842854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f251f-6e9a-47dd-9ada-27512519fd46 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:44.874156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 49/101] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:59:44.875519 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00018e85-732c-4708-977f-5ab4be006b0c None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:44.875813 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00018e85-732c-4708-977f-5ab4be006b0c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:44.876091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00018e85-732c-4708-977f-5ab4be006b0c None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:44.877476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 50/102] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:44.888493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cf19b-c59f-46ea-b40c-4f3bc06d21c6 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:44.888787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cf19b-c59f-46ea-b40c-4f3bc06d21c6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:44.889072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cf19b-c59f-46ea-b40c-4f3bc06d21c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:44.912902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8b9cf19b-c59f-46ea-b40c-4f3bc06d21c6 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:44.914539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8b9cf19b-c59f-46ea-b40c-4f3bc06d21c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:44.965858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f251f-6e9a-47dd-9ada-27512519fd46 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:44.992283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 51/103] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:59:44.993247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cf19b-c59f-46ea-b40c-4f3bc06d21c6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:45.012300 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171b0c7-c3d3-4988-bbb1-182c0eba9cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:45.013109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171b0c7-c3d3-4988-bbb1-182c0eba9cde None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:45.013390 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171b0c7-c3d3-4988-bbb1-182c0eba9cde None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:45.013951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171b0c7-c3d3-4988-bbb1-182c0eba9cde None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:45.016643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:45.027456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171b0c7-c3d3-4988-bbb1-182c0eba9cde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-c171b0c7-c3d3-4988-bbb1-182c0eba9cde None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:45.035268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:45.036122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:45.036919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:45.036919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:45.036919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:45.036919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:45.036919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:45.036919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:45.036919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:45.036919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:45.036919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:45.036919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:45.036919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:59:45.036919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 14:59:45.036919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 53/104] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:59:45.038084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26e9d1-f9a1-49ba-8d07-620f1a7dac53 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:45.038260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26e9d1-f9a1-49ba-8d07-620f1a7dac53 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:45.039424 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26e9d1-f9a1-49ba-8d07-620f1a7dac53 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:45.041903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 54/105] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:45.043400 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef98622-9549-477b-95da-fce1696f1bf2 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:45.043956 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef98622-9549-477b-95da-fce1696f1bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:45.044463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef98622-9549-477b-95da-fce1696f1bf2 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:45.046899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 55/106] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:45.046899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef98622-9549-477b-95da-fce1696f1bf2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:45.095033 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:45.110006 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cf19b-c59f-46ea-b40c-4f3bc06d21c6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:45.146899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad4b73-7fa1-4277-943f-de220d91fa91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:45.148400 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad4b73-7fa1-4277-943f-de220d91fa91 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:45.148693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad4b73-7fa1-4277-943f-de220d91fa91 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:45.148971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad4b73-7fa1-4277-943f-de220d91fa91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:45.150159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad4b73-7fa1-4277-943f-de220d91fa91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:45.151635 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad4b73-7fa1-4277-943f-de220d91fa91 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:45.153262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 52/107] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 14:59:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:45.154164 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad4b73-7fa1-4277-943f-de220d91fa91 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:45.178384 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef98622-9549-477b-95da-fce1696f1bf2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:45.218365 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153eb5c6-bb0c-40d8-bf97-ef5405ecb218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:45.219463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153eb5c6-bb0c-40d8-bf97-ef5405ecb218 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:45.220003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153eb5c6-bb0c-40d8-bf97-ef5405ecb218 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:45.220425 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153eb5c6-bb0c-40d8-bf97-ef5405ecb218 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:45.231105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153eb5c6-bb0c-40d8-bf97-ef5405ecb218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-153eb5c6-bb0c-40d8-bf97-ef5405ecb218 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:45.239402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:45.240546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:45.242083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:45.242083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:45.242083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:45.242083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:45.242083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:45.242083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:45.242083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:45.242083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:45.242083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:45.242083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:45.242083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:45.242083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 14:59:45.244231 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 56/108] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 14:59:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:45.245693 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091270a5-5050-410c-9b5c-39471a24b0a6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:45.245982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091270a5-5050-410c-9b5c-39471a24b0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:45.246261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091270a5-5050-410c-9b5c-39471a24b0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:45.265507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-091270a5-5050-410c-9b5c-39471a24b0a6 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:45.265680 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad4b73-7fa1-4277-943f-de220d91fa91 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:45.266712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-091270a5-5050-410c-9b5c-39471a24b0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:45.301315 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e939a-d6e4-4ea8-8387-08c4f2d01e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:45.301988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e939a-d6e4-4ea8-8387-08c4f2d01e1c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:45.302291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e939a-d6e4-4ea8-8387-08c4f2d01e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:45.302737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e939a-d6e4-4ea8-8387-08c4f2d01e1c None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:45.305102 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e939a-d6e4-4ea8-8387-08c4f2d01e1c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:45.305430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e939a-d6e4-4ea8-8387-08c4f2d01e1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:45.306825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e939a-d6e4-4ea8-8387-08c4f2d01e1c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:45.316428 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 53/109] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 14:59:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:45.317812 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79a14b-9ef4-4106-8cc1-4f0b861fcbe2 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:45.318243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79a14b-9ef4-4106-8cc1-4f0b861fcbe2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:45.318661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79a14b-9ef4-4106-8cc1-4f0b861fcbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:45.325111 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:45.337182 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3e79a14b-9ef4-4106-8cc1-4f0b861fcbe2 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:45.348529 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3e79a14b-9ef4-4106-8cc1-4f0b861fcbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:45.348647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 57/110] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:45.348647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091270a5-5050-410c-9b5c-39471a24b0a6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:45.353366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:45.423203 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 54/111] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:45.423203 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79a14b-9ef4-4106-8cc1-4f0b861fcbe2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:45.469290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091270a5-5050-410c-9b5c-39471a24b0a6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:45.503302 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b481c-7cd6-42c3-9dda-c8c44f9c7b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:45.504077 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b481c-7cd6-42c3-9dda-c8c44f9c7b51 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:45.504379 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b481c-7cd6-42c3-9dda-c8c44f9c7b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:45.504661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b481c-7cd6-42c3-9dda-c8c44f9c7b51 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:45.505374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b481c-7cd6-42c3-9dda-c8c44f9c7b51 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:45.507535 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b481c-7cd6-42c3-9dda-c8c44f9c7b51 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:45.513453 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79a14b-9ef4-4106-8cc1-4f0b861fcbe2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:45.548271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1893fb-a160-45a3-a37f-2d5fa84b6e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:45.548951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1893fb-a160-45a3-a37f-2d5fa84b6e91 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:45.549235 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1893fb-a160-45a3-a37f-2d5fa84b6e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:45.549513 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1893fb-a160-45a3-a37f-2d5fa84b6e91 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:45.550256 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1893fb-a160-45a3-a37f-2d5fa84b6e91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:45.550566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1893fb-a160-45a3-a37f-2d5fa84b6e91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:45.551978 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1893fb-a160-45a3-a37f-2d5fa84b6e91 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:45.560403 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 55/112] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:45.561220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1893fb-a160-45a3-a37f-2d5fa84b6e91 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:45.573756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:45.607504 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 58/113] 10.222.0.67 () {60 vars in 1141 bytes} [Fri Jun 24 14:59:45 2022] POST /identity/v3/projects => generated 302 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:59:45.612235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b481c-7cd6-42c3-9dda-c8c44f9c7b51 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:45.667455 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:45.693616 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1893fb-a160-45a3-a37f-2d5fa84b6e91 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:45.739457 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e6eb8-bbef-466c-9e58-4ec991348033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:45.740327 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e6eb8-bbef-466c-9e58-4ec991348033 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:45.740615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e6eb8-bbef-466c-9e58-4ec991348033 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:45.740937 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e6eb8-bbef-466c-9e58-4ec991348033 None admin] PATH_INFO: `/v3/users/aa3ffd98f5ef494789e283200d304e9f` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:45.758627 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e6eb8-bbef-466c-9e58-4ec991348033 None admin] RBAC: Authorizing `identity:get_user(user_id=aa3ffd98f5ef494789e283200d304e9f)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:45.760247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e6eb8-bbef-466c-9e58-4ec991348033 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:45.762118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 56/114] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:45 2022] GET /identity/v3/users/aa3ffd98f5ef494789e283200d304e9f => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:45.762890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e6eb8-bbef-466c-9e58-4ec991348033 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:45.764742 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b481c-7cd6-42c3-9dda-c8c44f9c7b51 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:45.840047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69e53a-9476-4060-a95b-93560e8060f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:45.840862 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69e53a-9476-4060-a95b-93560e8060f7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:45.841151 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69e53a-9476-4060-a95b-93560e8060f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:45.841435 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69e53a-9476-4060-a95b-93560e8060f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:45.853901 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69e53a-9476-4060-a95b-93560e8060f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ac69e53a-9476-4060-a95b-93560e8060f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:45.864160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:45.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:45.865868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:45.865868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:45.865868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:45.865868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:45.865868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:45.865868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:45.865868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:45.865868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:45.865868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:45.865868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:45.865868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:59:45.865868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 14:59:45.872755 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 59/115] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:45.872755 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69e53a-9476-4060-a95b-93560e8060f7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:45.904290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:45.955302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e6eb8-bbef-466c-9e58-4ec991348033 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:45.980142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:45.986495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69e53a-9476-4060-a95b-93560e8060f7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:46.005565 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9b4f6-8db1-4819-9a66-9cff336f49fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:46.007504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9b4f6-8db1-4819-9a66-9cff336f49fa None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:46.008156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9b4f6-8db1-4819-9a66-9cff336f49fa None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:46.008633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9b4f6-8db1-4819-9a66-9cff336f49fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:46.023645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9b4f6-8db1-4819-9a66-9cff336f49fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:46.033461 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cde65-2ba5-4942-8291-c4ddeb7cbce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:46.034348 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cde65-2ba5-4942-8291-c4ddeb7cbce6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:46.034641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cde65-2ba5-4942-8291-c4ddeb7cbce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:46.034921 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cde65-2ba5-4942-8291-c4ddeb7cbce6 None admin] PATH_INFO: `/v3/users/4f372b5546df46a8836cf73269458aff` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-32e9b4f6-8db1-4819-9a66-9cff336f49fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:46.035380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:46.036213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:46.036971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:46.036971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:46.036971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:46.036971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:46.036971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:46.036971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:46.036971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:46.036971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:46.036971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:46.036971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:46.036971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:59:46.036971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 14:59:46.038312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5cde65-2ba5-4942-8291-c4ddeb7cbce6 None admin] RBAC: Authorizing `identity:get_user(user_id=4f372b5546df46a8836cf73269458aff)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:46.039666 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 57/116] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:46.040700 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9b4f6-8db1-4819-9a66-9cff336f49fa None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:46.043740 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5cde65-2ba5-4942-8291-c4ddeb7cbce6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:46.048253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 60/117] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:45 2022] GET /identity/v3/users/4f372b5546df46a8836cf73269458aff => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:46.050119 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cde65-2ba5-4942-8291-c4ddeb7cbce6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:46.199723 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cde65-2ba5-4942-8291-c4ddeb7cbce6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:46.207407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9b4f6-8db1-4819-9a66-9cff336f49fa None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:46.242031 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77d568-6fff-4b03-b2b5-171606a69022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:46.242711 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77d568-6fff-4b03-b2b5-171606a69022 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:46.243034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77d568-6fff-4b03-b2b5-171606a69022 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:46.243699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77d568-6fff-4b03-b2b5-171606a69022 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:46.244466 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77d568-6fff-4b03-b2b5-171606a69022 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:46.244780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77d568-6fff-4b03-b2b5-171606a69022 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:46.246175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77d568-6fff-4b03-b2b5-171606a69022 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:46.252724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:46.273003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 61/118] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 14:59:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:46.273003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77d568-6fff-4b03-b2b5-171606a69022 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:46.273298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e2bf8-a743-4605-8374-4f39e2f370a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:46.273298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e2bf8-a743-4605-8374-4f39e2f370a9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:46.273298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e2bf8-a743-4605-8374-4f39e2f370a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:46.273298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e2bf8-a743-4605-8374-4f39e2f370a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:46.300629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:46.355060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e2bf8-a743-4605-8374-4f39e2f370a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:46.356695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e2bf8-a743-4605-8374-4f39e2f370a9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:46.386236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77d568-6fff-4b03-b2b5-171606a69022 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:46.436447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe3c74-0fae-4ddb-aa27-0f4616ffd83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:46.437130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe3c74-0fae-4ddb-aa27-0f4616ffd83e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:46.437419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe3c74-0fae-4ddb-aa27-0f4616ffd83e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:46.438422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe3c74-0fae-4ddb-aa27-0f4616ffd83e None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:46.438551 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe3c74-0fae-4ddb-aa27-0f4616ffd83e None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:46.438778 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe3c74-0fae-4ddb-aa27-0f4616ffd83e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:46.440195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe3c74-0fae-4ddb-aa27-0f4616ffd83e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:46.452179 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 62/119] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 14:59:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:46.457920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe3c74-0fae-4ddb-aa27-0f4616ffd83e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:46.478540 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 58/120] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 14:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:59:46.485118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc780f-4d3f-4811-8aa0-064f82e534fd None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:46.485616 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc780f-4d3f-4811-8aa0-064f82e534fd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:46.486091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc780f-4d3f-4811-8aa0-064f82e534fd None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:46.487911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 59/121] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:46.489298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc780f-4d3f-4811-8aa0-064f82e534fd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:46.525021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:46.571465 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:46.586160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe3c74-0fae-4ddb-aa27-0f4616ffd83e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:46.601664 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc780f-4d3f-4811-8aa0-064f82e534fd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:46.625427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82b20f-52d9-4455-b731-7524d652eed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:46.626393 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82b20f-52d9-4455-b731-7524d652eed8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:46.626897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82b20f-52d9-4455-b731-7524d652eed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:46.627002 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82b20f-52d9-4455-b731-7524d652eed8 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:46.628257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82b20f-52d9-4455-b731-7524d652eed8 None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:46.629715 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82b20f-52d9-4455-b731-7524d652eed8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:46.631639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 63/122] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:46 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:46.632885 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82b20f-52d9-4455-b731-7524d652eed8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:46.643414 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b06e5-8590-42b8-ae9f-bd002a8f690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:46.644370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b06e5-8590-42b8-ae9f-bd002a8f690e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:46.644661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b06e5-8590-42b8-ae9f-bd002a8f690e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:46.644943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b06e5-8590-42b8-ae9f-bd002a8f690e None admin] PATH_INFO: `/v3/users/2bf0569100d34ba7939616d4bcef590a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:46.646141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b06e5-8590-42b8-ae9f-bd002a8f690e None admin] RBAC: Authorizing `identity:get_user(user_id=2bf0569100d34ba7939616d4bcef590a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:46.648346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b06e5-8590-42b8-ae9f-bd002a8f690e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:46.649990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 60/123] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:46 2022] GET /identity/v3/users/2bf0569100d34ba7939616d4bcef590a => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:46.652216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b06e5-8590-42b8-ae9f-bd002a8f690e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:46.740267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82b20f-52d9-4455-b731-7524d652eed8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:46.754563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:46.762941 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b06e5-8590-42b8-ae9f-bd002a8f690e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:46.781357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb76b88-01e7-4714-9b70-13f80aa66217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:46.782334 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb76b88-01e7-4714-9b70-13f80aa66217 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:46.782510 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb76b88-01e7-4714-9b70-13f80aa66217 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:46.782742 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb76b88-01e7-4714-9b70-13f80aa66217 None admin] PATH_INFO: `/v3/users/2bf0569100d34ba7939616d4bcef590a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:46.784089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb76b88-01e7-4714-9b70-13f80aa66217 None admin] RBAC: Authorizing `identity:get_user(user_id=2bf0569100d34ba7939616d4bcef590a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:46.785911 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb76b88-01e7-4714-9b70-13f80aa66217 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:46.787536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 64/124] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:46 2022] GET /identity/v3/users/2bf0569100d34ba7939616d4bcef590a => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:46.789268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce24994-83ad-42f8-bf30-c18dc82d2683 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:46.789550 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce24994-83ad-42f8-bf30-c18dc82d2683 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:46.789831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce24994-83ad-42f8-bf30-c18dc82d2683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:46.806526 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-dce24994-83ad-42f8-bf30-c18dc82d2683 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:46.807679 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6ef0f-6f0c-47d3-a19d-4a099e264618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:46.807802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dce24994-83ad-42f8-bf30-c18dc82d2683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:46.808641 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6ef0f-6f0c-47d3-a19d-4a099e264618 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:46.808932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6ef0f-6f0c-47d3-a19d-4a099e264618 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:46.809211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6ef0f-6f0c-47d3-a19d-4a099e264618 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:46.810416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6ef0f-6f0c-47d3-a19d-4a099e264618 None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:46.811896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6ef0f-6f0c-47d3-a19d-4a099e264618 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:46.818391 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 61/125] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:46 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:46.818391 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6ef0f-6f0c-47d3-a19d-4a099e264618 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:46.831108 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:46.887910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 65/126] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:46.888992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce24994-83ad-42f8-bf30-c18dc82d2683 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:46.932954 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6ef0f-6f0c-47d3-a19d-4a099e264618 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:46.970725 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201b7a9-7ee4-4689-9081-6c535abd5f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:46.971663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201b7a9-7ee4-4689-9081-6c535abd5f53 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:46.971933 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201b7a9-7ee4-4689-9081-6c535abd5f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:46.972209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201b7a9-7ee4-4689-9081-6c535abd5f53 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235/users/4f372b5546df46a8836cf73269458aff/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:46.974375 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201b7a9-7ee4-4689-9081-6c535abd5f53 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d31d318708e4e9caa66106ce370a235, user_id=4f372b5546df46a8836cf73269458aff, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:46.976513 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201b7a9-7ee4-4689-9081-6c535abd5f53 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:46.999849 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce24994-83ad-42f8-bf30-c18dc82d2683 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:47.033565 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64e92a-cd90-419e-a7c1-ed688a2e8950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:47.033715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 62/127] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 14:59:46 2022] PUT /identity/v3/projects/4d31d318708e4e9caa66106ce370a235/users/4f372b5546df46a8836cf73269458aff/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:59:47.034377 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64e92a-cd90-419e-a7c1-ed688a2e8950 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:47.034664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64e92a-cd90-419e-a7c1-ed688a2e8950 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:47.034942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64e92a-cd90-419e-a7c1-ed688a2e8950 None admin] PATH_INFO: `/v3/projects/05c0e94829d343e78450ed30009cccc0` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:47.036171 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64e92a-cd90-419e-a7c1-ed688a2e8950 None admin] RBAC: Authorizing `identity:get_project(project_id=05c0e94829d343e78450ed30009cccc0)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:47.037633 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64e92a-cd90-419e-a7c1-ed688a2e8950 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:47.040315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 66/128] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:46 2022] GET /identity/v3/projects/05c0e94829d343e78450ed30009cccc0 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:47.040913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201b7a9-7ee4-4689-9081-6c535abd5f53 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:47.041025 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64e92a-cd90-419e-a7c1-ed688a2e8950 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:47.070598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:47.071497 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:47.162721 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201b7a9-7ee4-4689-9081-6c535abd5f53 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:47.187310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64e92a-cd90-419e-a7c1-ed688a2e8950 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:47.205793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29434e7f-f289-4768-a12e-60a532f65614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:47.207233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29434e7f-f289-4768-a12e-60a532f65614 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:47.207798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29434e7f-f289-4768-a12e-60a532f65614 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:47.208476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29434e7f-f289-4768-a12e-60a532f65614 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:47.210807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29434e7f-f289-4768-a12e-60a532f65614 None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:47.213410 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29434e7f-f289-4768-a12e-60a532f65614 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:47.215880 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 63/129] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:47 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:47.216986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29434e7f-f289-4768-a12e-60a532f65614 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:47.231081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e0ebc-dc04-4d7c-b467-ceb858766885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:47.231896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e0ebc-dc04-4d7c-b467-ceb858766885 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:47.232209 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e0ebc-dc04-4d7c-b467-ceb858766885 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:47.232794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e0ebc-dc04-4d7c-b467-ceb858766885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:47.233670 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e0ebc-dc04-4d7c-b467-ceb858766885 None admin] RBAC: Adding query filter params (role.id=ae37f175be764624b8ef6f3c0104dbf9, scope.project.id=4d31d318708e4e9caa66106ce370a235, user.id=aa3ffd98f5ef494789e283200d304e9f) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:47.233992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e0ebc-dc04-4d7c-b467-ceb858766885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:47.236057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e0ebc-dc04-4d7c-b467-ceb858766885 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:47.242797 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 67/130] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 14:59:47 2022] GET /identity/v3/role_assignments?user.id=aa3ffd98f5ef494789e283200d304e9f&scope.project.id=4d31d318708e4e9caa66106ce370a235&role.id=ae37f175be764624b8ef6f3c0104dbf9 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:47.243635 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e0ebc-dc04-4d7c-b467-ceb858766885 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:47.364538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:47.378769 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29434e7f-f289-4768-a12e-60a532f65614 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:47.391610 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:47.423500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f3968-3d8e-4e04-beb9-be6187f942cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:47.424580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f3968-3d8e-4e04-beb9-be6187f942cb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:47.425010 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f3968-3d8e-4e04-beb9-be6187f942cb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:47.425460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f3968-3d8e-4e04-beb9-be6187f942cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:47.438467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f3968-3d8e-4e04-beb9-be6187f942cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:47.441389 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e0ebc-dc04-4d7c-b467-ceb858766885 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f70f3968-3d8e-4e04-beb9-be6187f942cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:47.450419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:47.451354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:47.452159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:47.452159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:47.452159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:47.452159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:47.452159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:47.452159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:47.452159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:47.452159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:47.452159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:47.452159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:47.452159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:59:47.452159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 14:59:47.455431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 64/131] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 14:59:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:47.456419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f3968-3d8e-4e04-beb9-be6187f942cb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:47.516521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f358f-7a12-448d-a2c1-20738a12fcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:47.517449 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f358f-7a12-448d-a2c1-20738a12fcff None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:47.517734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f358f-7a12-448d-a2c1-20738a12fcff None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:47.518023 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f358f-7a12-448d-a2c1-20738a12fcff None admin] PATH_INFO: `/v3/projects/05c0e94829d343e78450ed30009cccc0/users/2bf0569100d34ba7939616d4bcef590a/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:47.520040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f358f-7a12-448d-a2c1-20738a12fcff None admin] RBAC: Authorizing `identity:create_grant(project_id=05c0e94829d343e78450ed30009cccc0, user_id=2bf0569100d34ba7939616d4bcef590a, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:47.521501 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f358f-7a12-448d-a2c1-20738a12fcff None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:47.583856 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f3968-3d8e-4e04-beb9-be6187f942cb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:47.622534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da2433-be79-44d4-8070-eced726865df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:47.623876 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da2433-be79-44d4-8070-eced726865df None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:47.624446 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da2433-be79-44d4-8070-eced726865df None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:47.624888 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da2433-be79-44d4-8070-eced726865df None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235/users/2bf0569100d34ba7939616d4bcef590a/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:47.627317 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da2433-be79-44d4-8070-eced726865df None admin] RBAC: Authorizing `identity:create_grant(project_id=4d31d318708e4e9caa66106ce370a235, user_id=2bf0569100d34ba7939616d4bcef590a, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:47.629029 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da2433-be79-44d4-8070-eced726865df None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:47.652504 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 68/132] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 14:59:47 2022] PUT /identity/v3/projects/05c0e94829d343e78450ed30009cccc0/users/2bf0569100d34ba7939616d4bcef590a/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:59:47.653905 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f358f-7a12-448d-a2c1-20738a12fcff None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:47.703339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 65/133] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 14:59:47 2022] PUT /identity/v3/projects/4d31d318708e4e9caa66106ce370a235/users/2bf0569100d34ba7939616d4bcef590a/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:59:47.767724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a909e-6c81-4d1c-aead-1002fa81a711 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:47.768805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a909e-6c81-4d1c-aead-1002fa81a711 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:47.769314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a909e-6c81-4d1c-aead-1002fa81a711 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:47.771278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 66/134] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:59:47.777928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a71b06-471b-4a28-8205-2f6e1b321002 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:47.778277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a71b06-471b-4a28-8205-2f6e1b321002 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:47.778609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a71b06-471b-4a28-8205-2f6e1b321002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:47.792408 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f358f-7a12-448d-a2c1-20738a12fcff None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:47.797919 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-76a71b06-471b-4a28-8205-2f6e1b321002 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:47.799335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-76a71b06-471b-4a28-8205-2f6e1b321002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:47.808920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:47.860080 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8ab60-c864-46ea-af69-6638f85a6a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:47.860853 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8ab60-c864-46ea-af69-6638f85a6a92 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:47.861146 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8ab60-c864-46ea-af69-6638f85a6a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:47.861487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8ab60-c864-46ea-af69-6638f85a6a92 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:47.862277 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8ab60-c864-46ea-af69-6638f85a6a92 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:47.862879 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8ab60-c864-46ea-af69-6638f85a6a92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:47.864855 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8ab60-c864-46ea-af69-6638f85a6a92 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:47.883096 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 69/135] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 14:59:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:47.894456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8ab60-c864-46ea-af69-6638f85a6a92 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:47.913547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:47.936596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 67/136] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:59:47.966231 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a71b06-471b-4a28-8205-2f6e1b321002 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:48.049335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8ab60-c864-46ea-af69-6638f85a6a92 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:48.112453 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c516cc4-3abc-4320-bb3d-5490bc6fd5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:48.113332 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c516cc4-3abc-4320-bb3d-5490bc6fd5ac None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:48.113618 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c516cc4-3abc-4320-bb3d-5490bc6fd5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:48.113900 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c516cc4-3abc-4320-bb3d-5490bc6fd5ac None admin] PATH_INFO: `/v3/users/2bf0569100d34ba7939616d4bcef590a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:48.117112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a71b06-471b-4a28-8205-2f6e1b321002 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:48.119361 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c516cc4-3abc-4320-bb3d-5490bc6fd5ac None admin] RBAC: Authorizing `identity:get_user(user_id=2bf0569100d34ba7939616d4bcef590a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:48.120841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c516cc4-3abc-4320-bb3d-5490bc6fd5ac None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:48.127824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 70/137] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:47 2022] GET /identity/v3/users/2bf0569100d34ba7939616d4bcef590a => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:48.129086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c516cc4-3abc-4320-bb3d-5490bc6fd5ac None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:48.187271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfcf59-e805-46b2-8876-cacc058d0f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:48.188009 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfcf59-e805-46b2-8876-cacc058d0f11 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:48.188395 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfcf59-e805-46b2-8876-cacc058d0f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:48.188679 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfcf59-e805-46b2-8876-cacc058d0f11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:48.189889 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfcf59-e805-46b2-8876-cacc058d0f11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:48.191387 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfcf59-e805-46b2-8876-cacc058d0f11 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:48.193066 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 68/138] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 14:59:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:48.198762 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfcf59-e805-46b2-8876-cacc058d0f11 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:48.214113 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:48.246488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:48.276249 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c516cc4-3abc-4320-bb3d-5490bc6fd5ac None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:48.332169 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342aeee0-3506-4b1a-a0d5-baaf655c6308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:48.333066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342aeee0-3506-4b1a-a0d5-baaf655c6308 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:48.333837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342aeee0-3506-4b1a-a0d5-baaf655c6308 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:48.334198 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342aeee0-3506-4b1a-a0d5-baaf655c6308 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:48.339987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342aeee0-3506-4b1a-a0d5-baaf655c6308 None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:48.341456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342aeee0-3506-4b1a-a0d5-baaf655c6308 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:48.343542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 71/139] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:48 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:48.351213 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342aeee0-3506-4b1a-a0d5-baaf655c6308 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:48.421038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfcf59-e805-46b2-8876-cacc058d0f11 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:48.475829 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342aeee0-3506-4b1a-a0d5-baaf655c6308 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:48.488436 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1853d-7766-49f3-bae2-ba48dd0c9617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:48.489551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1853d-7766-49f3-bae2-ba48dd0c9617 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:48.490084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1853d-7766-49f3-bae2-ba48dd0c9617 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:48.490557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1853d-7766-49f3-bae2-ba48dd0c9617 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:48.492044 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1853d-7766-49f3-bae2-ba48dd0c9617 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:48.494051 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1853d-7766-49f3-bae2-ba48dd0c9617 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:48.514079 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c093d-c883-4e8c-a05a-f4ecdb579321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:48.515033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c093d-c883-4e8c-a05a-f4ecdb579321 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:48.515355 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c093d-c883-4e8c-a05a-f4ecdb579321 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:48.515646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c093d-c883-4e8c-a05a-f4ecdb579321 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235/users/2bf0569100d34ba7939616d4bcef590a/roles/c077ebdf7f214d7585dfc1570eecf882` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:48.517967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c093d-c883-4e8c-a05a-f4ecdb579321 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d31d318708e4e9caa66106ce370a235, user_id=2bf0569100d34ba7939616d4bcef590a, role_id=c077ebdf7f214d7585dfc1570eecf882)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:48.519454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c093d-c883-4e8c-a05a-f4ecdb579321 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:48.599611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 69/140] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 14:59:48 2022] POST /identity/v3/users => generated 283 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:59:48.611412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 72/141] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 14:59:48 2022] PUT /identity/v3/projects/4d31d318708e4e9caa66106ce370a235/users/2bf0569100d34ba7939616d4bcef590a/roles/c077ebdf7f214d7585dfc1570eecf882 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:59:49.895210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba59615-c655-4609-80a3-6a55c4748008 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:49.895720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba59615-c655-4609-80a3-6a55c4748008 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:49.895794 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba59615-c655-4609-80a3-6a55c4748008 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:49.897378 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 70/142] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:49.902812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee23024-929d-44e6-9ab5-64570ed0aa95 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:49.903295 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee23024-929d-44e6-9ab5-64570ed0aa95 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:49.903469 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee23024-929d-44e6-9ab5-64570ed0aa95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:49.910581 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:49.922512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-aee23024-929d-44e6-9ab5-64570ed0aa95 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:49.923715 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-aee23024-929d-44e6-9ab5-64570ed0aa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:50.012537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 73/143] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:50.033341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27d1dd-7bd1-4f5f-888b-836bb642fe40 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:50.033930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27d1dd-7bd1-4f5f-888b-836bb642fe40 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:50.034414 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27d1dd-7bd1-4f5f-888b-836bb642fe40 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:50.036328 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 71/144] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:50.042502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c7554-16ce-4c31-b437-7bd0ff8ffd9c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:50.042791 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c7554-16ce-4c31-b437-7bd0ff8ffd9c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:50.043070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c7554-16ce-4c31-b437-7bd0ff8ffd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:50.057930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27d1dd-7bd1-4f5f-888b-836bb642fe40 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:50.063225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-110c7554-16ce-4c31-b437-7bd0ff8ffd9c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:50.063479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-110c7554-16ce-4c31-b437-7bd0ff8ffd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:50.185600 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:50.202174 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 74/145] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:50.230847 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c7554-16ce-4c31-b437-7bd0ff8ffd9c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:50.247957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27d1dd-7bd1-4f5f-888b-836bb642fe40 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:50.302609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209dc818-a2e8-42fc-bd90-4111a0d140c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:50.304427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dc818-a2e8-42fc-bd90-4111a0d140c1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:50.304913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dc818-a2e8-42fc-bd90-4111a0d140c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:50.305385 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dc818-a2e8-42fc-bd90-4111a0d140c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:50.327541 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209dc818-a2e8-42fc-bd90-4111a0d140c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-209dc818-a2e8-42fc-bd90-4111a0d140c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:50.337991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:50.338910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:50.339728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:50.339728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:50.339728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:50.339728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:50.339728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:50.339728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:50.339728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:50.339728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:50.339728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:50.339728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:50.339728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:50.339728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 14:59:50.347557 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:50.351378 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 72/146] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 14:59:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:59:50.355584 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209dc818-a2e8-42fc-bd90-4111a0d140c1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:50.411280 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c7554-16ce-4c31-b437-7bd0ff8ffd9c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:50.473750 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0619aaf-2b3c-4c88-b303-603e0eb78ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:50.474878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0619aaf-2b3c-4c88-b303-603e0eb78ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:50.475955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0619aaf-2b3c-4c88-b303-603e0eb78ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:50.481811 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0619aaf-2b3c-4c88-b303-603e0eb78ca7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:50.501807 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0619aaf-2b3c-4c88-b303-603e0eb78ca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a0619aaf-2b3c-4c88-b303-603e0eb78ca7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:50.517152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:50.518052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:50.518833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:50.518833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:50.518833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:50.518833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:50.518833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:50.518833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:50.518833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:50.518833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:50.518833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:50.518833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:50.518833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:59:50.518833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 14:59:50.524672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 75/147] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:59:50.530909 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0619aaf-2b3c-4c88-b303-603e0eb78ca7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:50.539141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209dc818-a2e8-42fc-bd90-4111a0d140c1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:50.582770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:50.608142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd569a20-7130-48e4-b7f3-0c1fe532814c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:50.609176 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd569a20-7130-48e4-b7f3-0c1fe532814c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:50.609657 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd569a20-7130-48e4-b7f3-0c1fe532814c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:50.610126 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd569a20-7130-48e4-b7f3-0c1fe532814c None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:50.611115 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd569a20-7130-48e4-b7f3-0c1fe532814c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:50.611710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd569a20-7130-48e4-b7f3-0c1fe532814c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:50.613347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd569a20-7130-48e4-b7f3-0c1fe532814c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:50.629941 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 73/148] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:50.636216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd569a20-7130-48e4-b7f3-0c1fe532814c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:50.715810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0619aaf-2b3c-4c88-b303-603e0eb78ca7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:50.758443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:50.781676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282d227-b21b-460b-84be-cbb5058eb383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:50.782685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282d227-b21b-460b-84be-cbb5058eb383 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:50.783166 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282d227-b21b-460b-84be-cbb5058eb383 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:50.783659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282d227-b21b-460b-84be-cbb5058eb383 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:50.784627 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282d227-b21b-460b-84be-cbb5058eb383 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:50.785149 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282d227-b21b-460b-84be-cbb5058eb383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:50.786719 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282d227-b21b-460b-84be-cbb5058eb383 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:50.800805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 76/149] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 14:59:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:50.807583 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282d227-b21b-460b-84be-cbb5058eb383 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:50.816040 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd569a20-7130-48e4-b7f3-0c1fe532814c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:50.905288 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ccf11-732f-46cc-901f-5c7ac1e05685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:50.906986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ccf11-732f-46cc-901f-5c7ac1e05685 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:50.907490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ccf11-732f-46cc-901f-5c7ac1e05685 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:50.907981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ccf11-732f-46cc-901f-5c7ac1e05685 None admin] PATH_INFO: `/v3/users/4f372b5546df46a8836cf73269458aff` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:50.911928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ccf11-732f-46cc-901f-5c7ac1e05685 None admin] RBAC: Authorizing `identity:get_user(user_id=4f372b5546df46a8836cf73269458aff)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:50.913876 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ccf11-732f-46cc-901f-5c7ac1e05685 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:50.915806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 74/150] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:50 2022] GET /identity/v3/users/4f372b5546df46a8836cf73269458aff => generated 247 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:50.921226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ccf11-732f-46cc-901f-5c7ac1e05685 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:50.981772 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282d227-b21b-460b-84be-cbb5058eb383 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:51.022004 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:51.024039 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bf4d5-ad22-4336-bac8-bc62920cf05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:51.025052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bf4d5-ad22-4336-bac8-bc62920cf05b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:51.025257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bf4d5-ad22-4336-bac8-bc62920cf05b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:51.025538 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bf4d5-ad22-4336-bac8-bc62920cf05b None admin] PATH_INFO: `/v3/users/aa3ffd98f5ef494789e283200d304e9f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:51.026768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bf4d5-ad22-4336-bac8-bc62920cf05b None admin] RBAC: Authorizing `identity:get_user(user_id=aa3ffd98f5ef494789e283200d304e9f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:51.028321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bf4d5-ad22-4336-bac8-bc62920cf05b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:51.030057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 77/151] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:50 2022] GET /identity/v3/users/aa3ffd98f5ef494789e283200d304e9f => generated 289 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:51.031372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d19dd-113e-40f3-98b8-882ab7b3e9d8 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:51.031659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d19dd-113e-40f3-98b8-882ab7b3e9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:51.032064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d19dd-113e-40f3-98b8-882ab7b3e9d8 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:51.039475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 78/152] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:51.039475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d19dd-113e-40f3-98b8-882ab7b3e9d8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:51.116807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ccf11-732f-46cc-901f-5c7ac1e05685 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:51.127121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:51.206020 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d19dd-113e-40f3-98b8-882ab7b3e9d8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:51.223214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851edfdd-971e-4a70-a54b-1cac74f6bc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:51.224215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851edfdd-971e-4a70-a54b-1cac74f6bc50 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:51.224685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851edfdd-971e-4a70-a54b-1cac74f6bc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:51.225145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851edfdd-971e-4a70-a54b-1cac74f6bc50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:51.271854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a8492-298c-4709-a327-c9814721a2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:51.273449 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a8492-298c-4709-a327-c9814721a2ec None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:51.273675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a8492-298c-4709-a327-c9814721a2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:51.273954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a8492-298c-4709-a327-c9814721a2ec None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:51.280214 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a8492-298c-4709-a327-c9814721a2ec None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:51.281988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a8492-298c-4709-a327-c9814721a2ec None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:51.284406 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 79/153] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:51 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:51.286163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21deeeb-5e9c-412b-9c8a-a492e701bdf2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:51.286454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21deeeb-5e9c-412b-9c8a-a492e701bdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:51.286739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21deeeb-5e9c-412b-9c8a-a492e701bdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:51.355090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a21deeeb-5e9c-412b-9c8a-a492e701bdf2 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:51.356926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a21deeeb-5e9c-412b-9c8a-a492e701bdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:51.412024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851edfdd-971e-4a70-a54b-1cac74f6bc50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:51.413939 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851edfdd-971e-4a70-a54b-1cac74f6bc50 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:51.481232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 80/154] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:51.482080 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21deeeb-5e9c-412b-9c8a-a492e701bdf2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:51.543773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:51.571679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:51.651714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21deeeb-5e9c-412b-9c8a-a492e701bdf2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:51.656908 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 75/155] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 14:59:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 736 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 14:59:51.657841 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851edfdd-971e-4a70-a54b-1cac74f6bc50 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:51.719937 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ad118-4fe5-4e10-aa45-3e58ad562408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:51.720864 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ad118-4fe5-4e10-aa45-3e58ad562408 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:51.721150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ad118-4fe5-4e10-aa45-3e58ad562408 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:51.721428 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ad118-4fe5-4e10-aa45-3e58ad562408 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235/users/aa3ffd98f5ef494789e283200d304e9f/roles/ae37f175be764624b8ef6f3c0104dbf9` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:51.724054 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ad118-4fe5-4e10-aa45-3e58ad562408 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d31d318708e4e9caa66106ce370a235, user_id=aa3ffd98f5ef494789e283200d304e9f, role_id=ae37f175be764624b8ef6f3c0104dbf9)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:51.726610 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ad118-4fe5-4e10-aa45-3e58ad562408 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:51.794820 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 81/156] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 14:59:51 2022] PUT /identity/v3/projects/4d31d318708e4e9caa66106ce370a235/users/aa3ffd98f5ef494789e283200d304e9f/roles/ae37f175be764624b8ef6f3c0104dbf9 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:59:51.796148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14829c-2214-4236-8c7b-22b4ab63c65c None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:51.796435 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14829c-2214-4236-8c7b-22b4ab63c65c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:51.796718 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14829c-2214-4236-8c7b-22b4ab63c65c None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:51.799262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 82/157] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:51.801482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14829c-2214-4236-8c7b-22b4ab63c65c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:51.809800 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851edfdd-971e-4a70-a54b-1cac74f6bc50 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:51.889891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a00fb0-f495-4f94-a65a-25fbeb767d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:51.890724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a00fb0-f495-4f94-a65a-25fbeb767d78 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:51.891008 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a00fb0-f495-4f94-a65a-25fbeb767d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:51.891319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a00fb0-f495-4f94-a65a-25fbeb767d78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:51.918772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a00fb0-f495-4f94-a65a-25fbeb767d78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-16a00fb0-f495-4f94-a65a-25fbeb767d78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:51.928168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:51.929829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:51.930729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:51.930729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:51.930729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:51.930729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:51.930729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:51.930729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:51.930729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:51.930729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:51.930729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:51.930729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:51.930729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:59:51.930729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 14:59:51.930729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 76/158] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:51.936940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311f35a-a7df-4d68-9d7a-2e029d8f65a7 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:51.937233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311f35a-a7df-4d68-9d7a-2e029d8f65a7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:51.937512 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311f35a-a7df-4d68-9d7a-2e029d8f65a7 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:51.939384 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 77/159] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:51.941612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d8375-9514-4582-8d8b-efdad42845cb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:51.941894 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d8375-9514-4582-8d8b-efdad42845cb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:51.942235 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d8375-9514-4582-8d8b-efdad42845cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:51.972318 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-585d8375-9514-4582-8d8b-efdad42845cb None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:51.974008 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-585d8375-9514-4582-8d8b-efdad42845cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:51.990430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:52.056405 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14829c-2214-4236-8c7b-22b4ab63c65c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:52.058563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 78/160] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:52.061662 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d8375-9514-4582-8d8b-efdad42845cb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:52.079719 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:52.132242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f585e6-66e4-46c8-8afa-4681a7d34dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:52.133172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f585e6-66e4-46c8-8afa-4681a7d34dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:52.133455 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f585e6-66e4-46c8-8afa-4681a7d34dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:52.133733 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f585e6-66e4-46c8-8afa-4681a7d34dd7 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:52.135150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f585e6-66e4-46c8-8afa-4681a7d34dd7 None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:52.136825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f585e6-66e4-46c8-8afa-4681a7d34dd7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:52.138579 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 83/161] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:51 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:52.139838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c00468-2df5-435b-bc56-737f599f3a20 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:52.140122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c00468-2df5-435b-bc56-737f599f3a20 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:52.140399 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c00468-2df5-435b-bc56-737f599f3a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:52.175945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d7c00468-2df5-435b-bc56-737f599f3a20 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:52.178471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d7c00468-2df5-435b-bc56-737f599f3a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:52.236494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d8375-9514-4582-8d8b-efdad42845cb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:52.289962 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 84/162] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:52.291324 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c00468-2df5-435b-bc56-737f599f3a20 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:52.328538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3eb4c-a1f0-4e2e-915d-b756ba904527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:52.329463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3eb4c-a1f0-4e2e-915d-b756ba904527 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:52.329752 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3eb4c-a1f0-4e2e-915d-b756ba904527 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:52.330031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3eb4c-a1f0-4e2e-915d-b756ba904527 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:52.331361 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3eb4c-a1f0-4e2e-915d-b756ba904527 None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:52.331361 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3eb4c-a1f0-4e2e-915d-b756ba904527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:52.332483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3eb4c-a1f0-4e2e-915d-b756ba904527 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:52.354951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 79/163] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 14:59:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:52.355873 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3eb4c-a1f0-4e2e-915d-b756ba904527 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:52.379624 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:52.483018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c00468-2df5-435b-bc56-737f599f3a20 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:52.499590 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:52.521212 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69a3a6-724a-413a-b087-b530c4554609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:52.522885 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69a3a6-724a-413a-b087-b530c4554609 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:52.523345 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69a3a6-724a-413a-b087-b530c4554609 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:52.523891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69a3a6-724a-413a-b087-b530c4554609 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:52.525710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3eb4c-a1f0-4e2e-915d-b756ba904527 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:52.536013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69a3a6-724a-413a-b087-b530c4554609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ee69a3a6-724a-413a-b087-b530c4554609 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:52.544657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:52.545573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:52.546459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:52.546459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:52.546459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:52.546459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:52.546459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:52.546459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:52.546459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:52.546459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:52.546459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:52.546459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:52.546459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:59:52.546459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 14:59:52.546459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 85/164] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:52.547931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69a3a6-724a-413a-b087-b530c4554609 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:52.632505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223985d-a468-4f75-8419-69e7d76cecc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:52.633095 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223985d-a468-4f75-8419-69e7d76cecc7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:52.633379 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223985d-a468-4f75-8419-69e7d76cecc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:52.633665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223985d-a468-4f75-8419-69e7d76cecc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:52.634501 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223985d-a468-4f75-8419-69e7d76cecc7 None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=4d31d318708e4e9caa66106ce370a235, user.id=4f372b5546df46a8836cf73269458aff) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:52.634813 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223985d-a468-4f75-8419-69e7d76cecc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:52.636287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223985d-a468-4f75-8419-69e7d76cecc7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:52.642448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 80/165] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 14:59:52 2022] GET /identity/v3/role_assignments?user.id=4f372b5546df46a8836cf73269458aff&scope.project.id=4d31d318708e4e9caa66106ce370a235&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:52.648774 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223985d-a468-4f75-8419-69e7d76cecc7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:52.658091 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69a3a6-724a-413a-b087-b530c4554609 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:52.698033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8adb8-127d-483a-a3a1-50778282a6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:52.698889 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8adb8-127d-483a-a3a1-50778282a6fe None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:52.699193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8adb8-127d-483a-a3a1-50778282a6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:52.699474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8adb8-127d-483a-a3a1-50778282a6fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:52.719547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8adb8-127d-483a-a3a1-50778282a6fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-b0d8adb8-127d-483a-a3a1-50778282a6fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:52.739105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:52.742592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:52.744644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:52.744644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:52.744644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:52.744644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:52.744644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:52.744644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:52.744644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:52.744644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:52.744644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:52.744644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:52.744644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:59:52.744644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 14:59:52.744644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 86/166] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 14:59:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:52.744644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8adb8-127d-483a-a3a1-50778282a6fe None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:52.805102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:52.816849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:52.868154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223985d-a468-4f75-8419-69e7d76cecc7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:52.901622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8adb8-127d-483a-a3a1-50778282a6fe None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:52.923937 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971b854-e5b7-4175-9458-a6090fe09258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:52.924834 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971b854-e5b7-4175-9458-a6090fe09258 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:52.925115 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971b854-e5b7-4175-9458-a6090fe09258 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:52.925390 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971b854-e5b7-4175-9458-a6090fe09258 None admin] PATH_INFO: `/v3/users/2bf0569100d34ba7939616d4bcef590a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:52.926655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971b854-e5b7-4175-9458-a6090fe09258 None admin] RBAC: Authorizing `identity:get_user(user_id=2bf0569100d34ba7939616d4bcef590a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:52.928121 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971b854-e5b7-4175-9458-a6090fe09258 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:52.930834 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 81/167] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:52 2022] GET /identity/v3/users/2bf0569100d34ba7939616d4bcef590a => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:52.936100 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971b854-e5b7-4175-9458-a6090fe09258 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:52.938374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81270d-7033-4c0a-ae92-c68e946d2df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:52.939211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81270d-7033-4c0a-ae92-c68e946d2df6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:52.939573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81270d-7033-4c0a-ae92-c68e946d2df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:52.939952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81270d-7033-4c0a-ae92-c68e946d2df6 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:52.940815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81270d-7033-4c0a-ae92-c68e946d2df6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:52.941268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81270d-7033-4c0a-ae92-c68e946d2df6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:52.942727 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81270d-7033-4c0a-ae92-c68e946d2df6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:52.952836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 87/168] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 14:59:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:52.953830 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81270d-7033-4c0a-ae92-c68e946d2df6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:53.062732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971b854-e5b7-4175-9458-a6090fe09258 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:53.073181 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:53.093556 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81270d-7033-4c0a-ae92-c68e946d2df6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:53.109307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306cec1-062c-46c8-a37a-e03a0ddd2ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:53.109993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306cec1-062c-46c8-a37a-e03a0ddd2ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:53.110276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306cec1-062c-46c8-a37a-e03a0ddd2ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:53.110548 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306cec1-062c-46c8-a37a-e03a0ddd2ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:53.111324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306cec1-062c-46c8-a37a-e03a0ddd2ac3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:53.111687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306cec1-062c-46c8-a37a-e03a0ddd2ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:53.113067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306cec1-062c-46c8-a37a-e03a0ddd2ac3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:53.120359 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:53.124806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 82/169] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 14:59:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:53.125495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306cec1-062c-46c8-a37a-e03a0ddd2ac3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:53.206484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaf6d9-247f-42dd-b072-4391812ae4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:53.207129 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaf6d9-247f-42dd-b072-4391812ae4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:53.207460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaf6d9-247f-42dd-b072-4391812ae4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:53.207744 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaf6d9-247f-42dd-b072-4391812ae4d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:53.234844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306cec1-062c-46c8-a37a-e03a0ddd2ac3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:53.274278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9250a4-ea18-4a2f-bdbb-6e5de1ae9912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:53.275171 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9250a4-ea18-4a2f-bdbb-6e5de1ae9912 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:53.275495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9250a4-ea18-4a2f-bdbb-6e5de1ae9912 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:53.275769 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9250a4-ea18-4a2f-bdbb-6e5de1ae9912 None admin] PATH_INFO: `/v3/users/2bf0569100d34ba7939616d4bcef590a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:53.276984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9250a4-ea18-4a2f-bdbb-6e5de1ae9912 None admin] RBAC: Authorizing `identity:get_user(user_id=2bf0569100d34ba7939616d4bcef590a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:53.279096 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9250a4-ea18-4a2f-bdbb-6e5de1ae9912 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:53.280784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 83/170] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:53 2022] GET /identity/v3/users/2bf0569100d34ba7939616d4bcef590a => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:53.281551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9250a4-ea18-4a2f-bdbb-6e5de1ae9912 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:53.371549 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:53.419209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9250a4-ea18-4a2f-bdbb-6e5de1ae9912 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:53.440278 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eaf6d9-247f-42dd-b072-4391812ae4d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:53.441851 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eaf6d9-247f-42dd-b072-4391812ae4d0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:53.486409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a056a8-7c6c-4c8b-add8-d5a044d93ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:53.487244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a056a8-7c6c-4c8b-add8-d5a044d93ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:53.487528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a056a8-7c6c-4c8b-add8-d5a044d93ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:53.487801 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a056a8-7c6c-4c8b-add8-d5a044d93ba5 None admin] PATH_INFO: `/v3/users/2bf0569100d34ba7939616d4bcef590a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:53.489117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a056a8-7c6c-4c8b-add8-d5a044d93ba5 None admin] RBAC: Authorizing `identity:get_user(user_id=2bf0569100d34ba7939616d4bcef590a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:53.490566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a056a8-7c6c-4c8b-add8-d5a044d93ba5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:53.492908 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 84/171] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:53 2022] GET /identity/v3/users/2bf0569100d34ba7939616d4bcef590a => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:53.493690 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a056a8-7c6c-4c8b-add8-d5a044d93ba5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:53.592594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:53.610172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 88/172] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 14:59:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:59:53.611495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f07130-701b-4551-b803-7220fde71932 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:53.611787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f07130-701b-4551-b803-7220fde71932 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:53.612062 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f07130-701b-4551-b803-7220fde71932 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:53.625788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 89/173] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:53.625788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f07130-701b-4551-b803-7220fde71932 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:53.641539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a056a8-7c6c-4c8b-add8-d5a044d93ba5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:53.684835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4daaf61-3fcd-47bb-ac91-365ec3453514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:53.685919 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4daaf61-3fcd-47bb-ac91-365ec3453514 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:53.686426 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4daaf61-3fcd-47bb-ac91-365ec3453514 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:53.686910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4daaf61-3fcd-47bb-ac91-365ec3453514 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:53.691876 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:53.801993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f07130-701b-4551-b803-7220fde71932 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:53.828006 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4daaf61-3fcd-47bb-ac91-365ec3453514 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:53.829504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4daaf61-3fcd-47bb-ac91-365ec3453514 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:53.838992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda6d44-a95a-4f3b-9c40-27e9e6fa4abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:53.839965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda6d44-a95a-4f3b-9c40-27e9e6fa4abe None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:53.840433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda6d44-a95a-4f3b-9c40-27e9e6fa4abe None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:53.840884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda6d44-a95a-4f3b-9c40-27e9e6fa4abe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:53.925109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:53.936037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda6d44-a95a-4f3b-9c40-27e9e6fa4abe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:53.937730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda6d44-a95a-4f3b-9c40-27e9e6fa4abe None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:53.961500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 85/174] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 14:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:59:53.963082 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4daaf61-3fcd-47bb-ac91-365ec3453514 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:54.002694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:54.052071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 90/175] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 14:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:59:54.058890 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e475e6-759b-4b20-8a19-4add06d8265a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:54.059264 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e475e6-759b-4b20-8a19-4add06d8265a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:54.059554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e475e6-759b-4b20-8a19-4add06d8265a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:54.072835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4daaf61-3fcd-47bb-ac91-365ec3453514 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:54.076793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-45e475e6-759b-4b20-8a19-4add06d8265a None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:54.077980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-45e475e6-759b-4b20-8a19-4add06d8265a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:54.114479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2303f0-9a8a-4b72-8386-816577633a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:54.115379 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2303f0-9a8a-4b72-8386-816577633a5c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:54.115659 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2303f0-9a8a-4b72-8386-816577633a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:54.116591 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2303f0-9a8a-4b72-8386-816577633a5c None admin] PATH_INFO: `/v3/projects/05c0e94829d343e78450ed30009cccc0` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:54.117771 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2303f0-9a8a-4b72-8386-816577633a5c None admin] RBAC: Authorizing `identity:get_project(project_id=05c0e94829d343e78450ed30009cccc0)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:54.119254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2303f0-9a8a-4b72-8386-816577633a5c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:54.125709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 86/176] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:53 2022] GET /identity/v3/projects/05c0e94829d343e78450ed30009cccc0 => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:54.125709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fe9e9-ee7f-4ad9-836d-db036aa45e19 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:54.125709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fe9e9-ee7f-4ad9-836d-db036aa45e19 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:54.125709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fe9e9-ee7f-4ad9-836d-db036aa45e19 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:54.125709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 87/177] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:54.125709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6fe9e9-ee7f-4ad9-836d-db036aa45e19 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:54.152010 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 91/178] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:54.152849 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e475e6-759b-4b20-8a19-4add06d8265a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:54.186577 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:54.248567 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6fe9e9-ee7f-4ad9-836d-db036aa45e19 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:54.251031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:54.267898 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e475e6-759b-4b20-8a19-4add06d8265a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:54.287385 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca07bbc-7029-4dba-b13c-b0330a5be11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:54.289174 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca07bbc-7029-4dba-b13c-b0330a5be11d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:54.289469 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca07bbc-7029-4dba-b13c-b0330a5be11d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:54.289748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca07bbc-7029-4dba-b13c-b0330a5be11d None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:54.290960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca07bbc-7029-4dba-b13c-b0330a5be11d None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:54.292449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca07bbc-7029-4dba-b13c-b0330a5be11d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:54.298613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 88/179] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:54 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:54.298613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5a248-d989-4639-9d9a-4a5f125ee660 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:54.298613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5a248-d989-4639-9d9a-4a5f125ee660 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:54.298613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5a248-d989-4639-9d9a-4a5f125ee660 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:54.298613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 89/180] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:54.298613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5a248-d989-4639-9d9a-4a5f125ee660 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:54.310907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14bcca0-d54b-449d-92ac-823e749521e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:54.311646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bcca0-d54b-449d-92ac-823e749521e1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:54.311931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bcca0-d54b-449d-92ac-823e749521e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:54.312209 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bcca0-d54b-449d-92ac-823e749521e1 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:54.313421 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14bcca0-d54b-449d-92ac-823e749521e1 None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:54.314817 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14bcca0-d54b-449d-92ac-823e749521e1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:54.316544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 92/181] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:54 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:54.317666 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158729-15e2-484a-969d-19b1b4c6943b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:54.317950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158729-15e2-484a-969d-19b1b4c6943b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:54.318227 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158729-15e2-484a-969d-19b1b4c6943b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:54.334708 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-de158729-15e2-484a-969d-19b1b4c6943b None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:54.335919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-de158729-15e2-484a-969d-19b1b4c6943b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:54.408124 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 93/182] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:54.409321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de158729-15e2-484a-969d-19b1b4c6943b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:54.430342 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5a248-d989-4639-9d9a-4a5f125ee660 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:54.431930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:54.473157 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a48f10-654e-4479-87db-fba71f02e8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:54.473923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a48f10-654e-4479-87db-fba71f02e8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:54.474211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a48f10-654e-4479-87db-fba71f02e8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:54.474484 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a48f10-654e-4479-87db-fba71f02e8f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:54.475891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a48f10-654e-4479-87db-fba71f02e8f9 None admin] RBAC: Adding query filter params (role.id=b6dd71d4043d43e9823ec25176863347, scope.project.id=05c0e94829d343e78450ed30009cccc0, user.id=2bf0569100d34ba7939616d4bcef590a) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:54.476210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a48f10-654e-4479-87db-fba71f02e8f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:54.477773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a48f10-654e-4479-87db-fba71f02e8f9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:54.483653 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 90/183] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 14:59:54 2022] GET /identity/v3/role_assignments?user.id=2bf0569100d34ba7939616d4bcef590a&scope.project.id=05c0e94829d343e78450ed30009cccc0&role.id=b6dd71d4043d43e9823ec25176863347 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:54.484455 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a48f10-654e-4479-87db-fba71f02e8f9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:54.526706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:54.548290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de158729-15e2-484a-969d-19b1b4c6943b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:54.590797 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7748b-0997-4a04-9e2f-c2930e7838dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:54.591614 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7748b-0997-4a04-9e2f-c2930e7838dd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:54.591952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7748b-0997-4a04-9e2f-c2930e7838dd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:54.592232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7748b-0997-4a04-9e2f-c2930e7838dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:54.611245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7748b-0997-4a04-9e2f-c2930e7838dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-22e7748b-0997-4a04-9e2f-c2930e7838dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:54.620372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:54.621349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:54.622173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:54.622173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:54.622173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:54.622173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:54.622173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:54.622173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:54.622173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:54.622173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:54.622173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:54.622173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:54.622173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:54.622173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 14:59:54.622811 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a48f10-654e-4479-87db-fba71f02e8f9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:54.622890 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 94/184] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 14:59:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:54.623689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d518dd-7d03-4dc9-a015-7a89c39f7382 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:54.623689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d518dd-7d03-4dc9-a015-7a89c39f7382 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:54.623982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d518dd-7d03-4dc9-a015-7a89c39f7382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:54.642311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d1d518dd-7d03-4dc9-a015-7a89c39f7382 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:54.645670 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d1d518dd-7d03-4dc9-a015-7a89c39f7382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:54.665089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ef737-730d-46c8-a48d-35ce14d10ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:54.665760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ef737-730d-46c8-a48d-35ce14d10ead None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:54.666046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ef737-730d-46c8-a48d-35ce14d10ead None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:54.666326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ef737-730d-46c8-a48d-35ce14d10ead None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:54.667334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ef737-730d-46c8-a48d-35ce14d10ead None admin] RBAC: Adding query filter params (role.id=b6dd71d4043d43e9823ec25176863347, scope.project.id=4d31d318708e4e9caa66106ce370a235, user.id=2bf0569100d34ba7939616d4bcef590a) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:54.667535 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ef737-730d-46c8-a48d-35ce14d10ead None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:54.669254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ef737-730d-46c8-a48d-35ce14d10ead None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:54.675631 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 91/185] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 14:59:54 2022] GET /identity/v3/role_assignments?user.id=2bf0569100d34ba7939616d4bcef590a&scope.project.id=4d31d318708e4e9caa66106ce370a235&role.id=b6dd71d4043d43e9823ec25176863347 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:54.675870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ef737-730d-46c8-a48d-35ce14d10ead None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:54.732626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 95/186] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:54.733545 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d518dd-7d03-4dc9-a015-7a89c39f7382 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:54.748142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:54.818414 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:54.822167 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ef737-730d-46c8-a48d-35ce14d10ead None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:54.863451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d518dd-7d03-4dc9-a015-7a89c39f7382 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:54.864893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebb1fc-b0f1-45ca-9b3a-24f0ab541454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:54.865567 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebb1fc-b0f1-45ca-9b3a-24f0ab541454 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:54.865920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebb1fc-b0f1-45ca-9b3a-24f0ab541454 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:54.866208 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebb1fc-b0f1-45ca-9b3a-24f0ab541454 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:54.867162 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ebb1fc-b0f1-45ca-9b3a-24f0ab541454 None admin] RBAC: Adding query filter params (role.id=c077ebdf7f214d7585dfc1570eecf882, scope.project.id=4d31d318708e4e9caa66106ce370a235, user.id=2bf0569100d34ba7939616d4bcef590a) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:54.867429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ebb1fc-b0f1-45ca-9b3a-24f0ab541454 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:54.868880 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ebb1fc-b0f1-45ca-9b3a-24f0ab541454 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:54.875157 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 92/187] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 14:59:54 2022] GET /identity/v3/role_assignments?user.id=2bf0569100d34ba7939616d4bcef590a&scope.project.id=4d31d318708e4e9caa66106ce370a235&role.id=c077ebdf7f214d7585dfc1570eecf882 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:54.880567 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07bd79-f7d6-4451-a3f3-0e9844e64600 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:54.880861 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07bd79-f7d6-4451-a3f3-0e9844e64600 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:54.881144 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07bd79-f7d6-4451-a3f3-0e9844e64600 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:54.883482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 93/188] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:54.884318 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07bd79-f7d6-4451-a3f3-0e9844e64600 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:54.923564 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b7959-b54c-48b7-9f51-702392f0acc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:54.924894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b7959-b54c-48b7-9f51-702392f0acc3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:54.925386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b7959-b54c-48b7-9f51-702392f0acc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:54.925851 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b7959-b54c-48b7-9f51-702392f0acc3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:54.939526 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b7959-b54c-48b7-9f51-702392f0acc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-888b7959-b54c-48b7-9f51-702392f0acc3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:54.947993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:54.948982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:54.950247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:54.950247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:54.950247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:54.950247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:54.950247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:54.950247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:54.950247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:54.950247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:54.950247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:54.950247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:54.950247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:59:54.950247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 14:59:54.950247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 96/189] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 14:59:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:54.951125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b7959-b54c-48b7-9f51-702392f0acc3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:55.020971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07bd79-f7d6-4451-a3f3-0e9844e64600 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:55.031753 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:55.062893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f9caa-c1cc-488c-848b-dec52af26941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:55.063619 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f9caa-c1cc-488c-848b-dec52af26941 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:55.063901 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f9caa-c1cc-488c-848b-dec52af26941 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:55.064179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f9caa-c1cc-488c-848b-dec52af26941 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:55.064926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f9caa-c1cc-488c-848b-dec52af26941 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:55.065235 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f9caa-c1cc-488c-848b-dec52af26941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:55.066607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f9caa-c1cc-488c-848b-dec52af26941 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:55.073163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b7959-b54c-48b7-9f51-702392f0acc3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:55.076623 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 94/190] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:55.078607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94823538-81dc-4435-9723-209cf26a0ffc None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:55.078607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94823538-81dc-4435-9723-209cf26a0ffc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:55.078607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94823538-81dc-4435-9723-209cf26a0ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:55.103212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-94823538-81dc-4435-9723-209cf26a0ffc None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:55.104510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-94823538-81dc-4435-9723-209cf26a0ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:55.120741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518e6c4-87ed-4fd3-b1bb-bcd81f3bb065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:55.121483 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518e6c4-87ed-4fd3-b1bb-bcd81f3bb065 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:55.121766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518e6c4-87ed-4fd3-b1bb-bcd81f3bb065 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:55.122042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518e6c4-87ed-4fd3-b1bb-bcd81f3bb065 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:55.123264 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518e6c4-87ed-4fd3-b1bb-bcd81f3bb065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:55.124733 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518e6c4-87ed-4fd3-b1bb-bcd81f3bb065 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:55.125492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:55.126374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 97/191] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 14:59:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:55.127269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518e6c4-87ed-4fd3-b1bb-bcd81f3bb065 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:55.172800 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 95/192] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:55.173628 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94823538-81dc-4435-9723-209cf26a0ffc None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:55.235803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518e6c4-87ed-4fd3-b1bb-bcd81f3bb065 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:55.268653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb674c62-5f2e-4b14-9d09-d4c0cca4b032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:55.269320 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb674c62-5f2e-4b14-9d09-d4c0cca4b032 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:55.269606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb674c62-5f2e-4b14-9d09-d4c0cca4b032 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:55.269884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb674c62-5f2e-4b14-9d09-d4c0cca4b032 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:55.270311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94823538-81dc-4435-9723-209cf26a0ffc None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:55.270633 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb674c62-5f2e-4b14-9d09-d4c0cca4b032 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:55.270942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb674c62-5f2e-4b14-9d09-d4c0cca4b032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:55.272941 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb674c62-5f2e-4b14-9d09-d4c0cca4b032 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:55.277907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:55.281168 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 98/193] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 14:59:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:55.281929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb674c62-5f2e-4b14-9d09-d4c0cca4b032 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:55.305363 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577d81e-74b7-4134-af52-32520a1d56a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:55.306240 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577d81e-74b7-4134-af52-32520a1d56a4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:55.306511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577d81e-74b7-4134-af52-32520a1d56a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:55.306794 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577d81e-74b7-4134-af52-32520a1d56a4 None admin] PATH_INFO: `/v3/users/cf20004de8ff441cb52689a8d2b9e0d8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:55.319744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577d81e-74b7-4134-af52-32520a1d56a4 None admin] RBAC: Authorizing `identity:get_user(user_id=cf20004de8ff441cb52689a8d2b9e0d8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:55.321346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577d81e-74b7-4134-af52-32520a1d56a4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:55.323142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 96/194] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:55 2022] GET /identity/v3/users/cf20004de8ff441cb52689a8d2b9e0d8 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:55.323759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577d81e-74b7-4134-af52-32520a1d56a4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:55.338678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:55.387808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb674c62-5f2e-4b14-9d09-d4c0cca4b032 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:55.420800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853393b-4aae-41b6-83a6-c9ca11c8dbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:55.421463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853393b-4aae-41b6-83a6-c9ca11c8dbe9 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:55.421575 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577d81e-74b7-4134-af52-32520a1d56a4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:55.421745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853393b-4aae-41b6-83a6-c9ca11c8dbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:55.422020 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853393b-4aae-41b6-83a6-c9ca11c8dbe9 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:55.422726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853393b-4aae-41b6-83a6-c9ca11c8dbe9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:55.424207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853393b-4aae-41b6-83a6-c9ca11c8dbe9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:55.455807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7a445-0dc8-4899-b801-8292721b9878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:55.456606 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7a445-0dc8-4899-b801-8292721b9878 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:55.456894 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7a445-0dc8-4899-b801-8292721b9878 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:55.457172 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7a445-0dc8-4899-b801-8292721b9878 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:55.467886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7a445-0dc8-4899-b801-8292721b9878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-3fa7a445-0dc8-4899-b801-8292721b9878 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:55.475798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:55.476755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:55.477562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:55.477562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:55.477562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:55.477562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:55.477562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:55.477562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:55.477562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:55.477562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:55.477562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:55.477562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:55.477562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:59:55.477562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 14:59:55.477562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 97/195] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:55.478235 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7a445-0dc8-4899-b801-8292721b9878 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:55.501614 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 99/196] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 14:59:55 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:59:55.501614 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853393b-4aae-41b6-83a6-c9ca11c8dbe9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:55.572438 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:55.575159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:55.603113 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7a445-0dc8-4899-b801-8292721b9878 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:55.625427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853393b-4aae-41b6-83a6-c9ca11c8dbe9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:55.641159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271faa9c-4d7d-4713-af90-e57db000a121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:55.641971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271faa9c-4d7d-4713-af90-e57db000a121 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:55.642254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271faa9c-4d7d-4713-af90-e57db000a121 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:55.642528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271faa9c-4d7d-4713-af90-e57db000a121 None admin] PATH_INFO: `/v3/users/cf20004de8ff441cb52689a8d2b9e0d8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:55.643845 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271faa9c-4d7d-4713-af90-e57db000a121 None admin] RBAC: Authorizing `identity:get_user(user_id=cf20004de8ff441cb52689a8d2b9e0d8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:55.645324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271faa9c-4d7d-4713-af90-e57db000a121 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:55.651673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 98/197] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:55 2022] GET /identity/v3/users/cf20004de8ff441cb52689a8d2b9e0d8 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:55.651673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271faa9c-4d7d-4713-af90-e57db000a121 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:55.670982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e9a30-0cb3-4fa8-905e-194c4fa2caeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:55.671682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e9a30-0cb3-4fa8-905e-194c4fa2caeb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:55.672674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e9a30-0cb3-4fa8-905e-194c4fa2caeb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:55.672674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e9a30-0cb3-4fa8-905e-194c4fa2caeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:55.747023 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e9a30-0cb3-4fa8-905e-194c4fa2caeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:55.748864 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e9a30-0cb3-4fa8-905e-194c4fa2caeb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:55.784626 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271faa9c-4d7d-4713-af90-e57db000a121 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:55.821477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:55.832707 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f8d74-573d-4761-801e-15ac862895ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:55.833662 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f8d74-573d-4761-801e-15ac862895ca None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:55.833662 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f8d74-573d-4761-801e-15ac862895ca None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:55.833940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f8d74-573d-4761-801e-15ac862895ca None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:55.838719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f8d74-573d-4761-801e-15ac862895ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:55.838719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f8d74-573d-4761-801e-15ac862895ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:55.838719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f8d74-573d-4761-801e-15ac862895ca None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:55.849977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 99/198] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 14:59:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:55.849977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f8d74-573d-4761-801e-15ac862895ca None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:55.854560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:55.886458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 100/199] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 14:59:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:59:55.887599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e9a30-0cb3-4fa8-905e-194c4fa2caeb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:55.977073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f8d74-573d-4761-801e-15ac862895ca None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:56.010421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae042c5-b4de-4ff2-8027-43548ba07c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:56.011078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae042c5-b4de-4ff2-8027-43548ba07c26 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:56.011440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae042c5-b4de-4ff2-8027-43548ba07c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:56.011717 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae042c5-b4de-4ff2-8027-43548ba07c26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:56.016203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e9a30-0cb3-4fa8-905e-194c4fa2caeb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:56.056083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39baaa26-f827-4d90-bd0b-b1de8588c6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:56.057371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baaa26-f827-4d90-bd0b-b1de8588c6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:56.057926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baaa26-f827-4d90-bd0b-b1de8588c6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:56.058405 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baaa26-f827-4d90-bd0b-b1de8588c6b8 None admin] PATH_INFO: `/v3/users/aa3ffd98f5ef494789e283200d304e9f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:56.059911 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39baaa26-f827-4d90-bd0b-b1de8588c6b8 None admin] RBAC: Authorizing `identity:get_user(user_id=aa3ffd98f5ef494789e283200d304e9f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:56.061625 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39baaa26-f827-4d90-bd0b-b1de8588c6b8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:56.063515 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 101/200] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:55 2022] GET /identity/v3/users/aa3ffd98f5ef494789e283200d304e9f => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:56.064623 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39baaa26-f827-4d90-bd0b-b1de8588c6b8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:56.083906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae042c5-b4de-4ff2-8027-43548ba07c26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:56.085920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae042c5-b4de-4ff2-8027-43548ba07c26 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:56.092686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:56.098150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:56.181470 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39baaa26-f827-4d90-bd0b-b1de8588c6b8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:56.219960 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7513990-10fd-4ad8-85ea-9f5e9f49db65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:56.221118 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7513990-10fd-4ad8-85ea-9f5e9f49db65 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:56.221571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7513990-10fd-4ad8-85ea-9f5e9f49db65 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:56.222097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7513990-10fd-4ad8-85ea-9f5e9f49db65 None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:56.223609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7513990-10fd-4ad8-85ea-9f5e9f49db65 None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:56.225364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7513990-10fd-4ad8-85ea-9f5e9f49db65 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:56.231586 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 102/201] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:56 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:56.232596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7513990-10fd-4ad8-85ea-9f5e9f49db65 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:56.237224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 100/202] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 14:59:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:59:56.239564 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae042c5-b4de-4ff2-8027-43548ba07c26 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:56.337530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:56.347525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7513990-10fd-4ad8-85ea-9f5e9f49db65 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:56.350634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae042c5-b4de-4ff2-8027-43548ba07c26 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:56.353083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:56.386484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddabca-3f86-429a-b28c-0b8c36e3b8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:56.386953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c939a2-997f-4bf4-b490-c161418f3d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:56.387652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c939a2-997f-4bf4-b490-c161418f3d8d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:56.387918 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddabca-3f86-429a-b28c-0b8c36e3b8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:56.388018 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c939a2-997f-4bf4-b490-c161418f3d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:56.388195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c939a2-997f-4bf4-b490-c161418f3d8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:56.388611 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddabca-3f86-429a-b28c-0b8c36e3b8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:56.389027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c939a2-997f-4bf4-b490-c161418f3d8d None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487, user.id=cf20004de8ff441cb52689a8d2b9e0d8) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:56.389190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddabca-3f86-429a-b28c-0b8c36e3b8e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:56.389326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c939a2-997f-4bf4-b490-c161418f3d8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:56.390905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c939a2-997f-4bf4-b490-c161418f3d8d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:56.396761 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 101/203] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 14:59:56 2022] GET /identity/v3/role_assignments?user.id=cf20004de8ff441cb52689a8d2b9e0d8&scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:56.404294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c939a2-997f-4bf4-b490-c161418f3d8d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:56.528193 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c939a2-997f-4bf4-b490-c161418f3d8d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:56.571723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638081e-e683-4400-9141-4ac8c5f3208c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:56.573017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638081e-e683-4400-9141-4ac8c5f3208c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:56.573529 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638081e-e683-4400-9141-4ac8c5f3208c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:56.574007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638081e-e683-4400-9141-4ac8c5f3208c None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:56.575546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638081e-e683-4400-9141-4ac8c5f3208c None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:56.578027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638081e-e683-4400-9141-4ac8c5f3208c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:56.580885 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 102/204] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:56 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:56.585286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddabca-3f86-429a-b28c-0b8c36e3b8e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:56.585286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddabca-3f86-429a-b28c-0b8c36e3b8e0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:56.587590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638081e-e683-4400-9141-4ac8c5f3208c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:56.624675 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:56.701926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638081e-e683-4400-9141-4ac8c5f3208c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:56.739477 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f4124-f8f5-4103-8405-8efb732f8567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:56.740155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f4124-f8f5-4103-8405-8efb732f8567 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:56.740442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f4124-f8f5-4103-8405-8efb732f8567 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:56.741230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f4124-f8f5-4103-8405-8efb732f8567 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:56.742453 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f4124-f8f5-4103-8405-8efb732f8567 None admin] RBAC: Adding query filter params (role.id=c077ebdf7f214d7585dfc1570eecf882, scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487, user.id=cf20004de8ff441cb52689a8d2b9e0d8) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:56.742775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f4124-f8f5-4103-8405-8efb732f8567 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:56.744406 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f4124-f8f5-4103-8405-8efb732f8567 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:56.745176 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:56.751292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 103/205] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 14:59:56 2022] GET /identity/v3/role_assignments?user.id=cf20004de8ff441cb52689a8d2b9e0d8&scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487&role.id=c077ebdf7f214d7585dfc1570eecf882 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:56.765982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 103/206] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 14:59:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:59:56.772686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f4124-f8f5-4103-8405-8efb732f8567 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:56.926797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:56.931060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f4124-f8f5-4103-8405-8efb732f8567 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:56.976640 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1db941-a85e-4261-9ee6-a99e187af907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:56.977862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1db941-a85e-4261-9ee6-a99e187af907 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:56.978346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1db941-a85e-4261-9ee6-a99e187af907 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:56.978847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1db941-a85e-4261-9ee6-a99e187af907 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:56.980451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1db941-a85e-4261-9ee6-a99e187af907 None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:56.982095 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1db941-a85e-4261-9ee6-a99e187af907 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:56.984341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 104/207] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:56 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:56.989825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddabca-3f86-429a-b28c-0b8c36e3b8e0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:57.101410 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddabca-3f86-429a-b28c-0b8c36e3b8e0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:57.154732 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c2818-cdb7-4db3-801b-bfb728b19beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:57.155784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c2818-cdb7-4db3-801b-bfb728b19beb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:57.156275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c2818-cdb7-4db3-801b-bfb728b19beb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:57.156796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c2818-cdb7-4db3-801b-bfb728b19beb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:57.157849 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c2818-cdb7-4db3-801b-bfb728b19beb None admin] RBAC: Adding query filter params (role.id=ae37f175be764624b8ef6f3c0104dbf9, scope.project.id=4d31d318708e4e9caa66106ce370a235, user.id=aa3ffd98f5ef494789e283200d304e9f) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:57.158368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c2818-cdb7-4db3-801b-bfb728b19beb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:57.160049 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c2818-cdb7-4db3-801b-bfb728b19beb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:57.171760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 104/208] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 14:59:56 2022] GET /identity/v3/role_assignments?user.id=aa3ffd98f5ef494789e283200d304e9f&scope.project.id=4d31d318708e4e9caa66106ce370a235&role.id=ae37f175be764624b8ef6f3c0104dbf9 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:57.796972 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a75fe5-8041-4467-b053-ba111dbb721c None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:57.797507 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a75fe5-8041-4467-b053-ba111dbb721c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:57.797507 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a75fe5-8041-4467-b053-ba111dbb721c None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:57.799168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 105/209] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:57.804856 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12b73a-c2a4-4602-bd08-b13ab674f4ac None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:57.805364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12b73a-c2a4-4602-bd08-b13ab674f4ac None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:57.805447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12b73a-c2a4-4602-bd08-b13ab674f4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:57.845861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7a12b73a-c2a4-4602-bd08-b13ab674f4ac None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:57.847064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7a12b73a-c2a4-4602-bd08-b13ab674f4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:57.909418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:57.926697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 105/210] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:59:57.951522 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a75fe5-8041-4467-b053-ba111dbb721c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:58.051765 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49af39-b05d-45e1-bc96-b519545938d7 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:58.051968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49af39-b05d-45e1-bc96-b519545938d7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:58.052491 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49af39-b05d-45e1-bc96-b519545938d7 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:58.054593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 106/211] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:58.059848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5c800-540b-404f-bb97-92d473ed9794 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:58.060126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5c800-540b-404f-bb97-92d473ed9794 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:58.060411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5c800-540b-404f-bb97-92d473ed9794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:58.062071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a75fe5-8041-4467-b053-ba111dbb721c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:58.081493 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8cc5c800-540b-404f-bb97-92d473ed9794 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:58.082683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8cc5c800-540b-404f-bb97-92d473ed9794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:58.107839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af55a7a-9a1d-4c16-bc04-1c35c2461c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:58.108948 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af55a7a-9a1d-4c16-bc04-1c35c2461c23 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:58.109428 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af55a7a-9a1d-4c16-bc04-1c35c2461c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:58.109920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af55a7a-9a1d-4c16-bc04-1c35c2461c23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:58.120966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af55a7a-9a1d-4c16-bc04-1c35c2461c23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2af55a7a-9a1d-4c16-bc04-1c35c2461c23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:58.129440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:58.130336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:58.131163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:58.131163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:58.131163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:58.131163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:58.131163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:58.131163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:58.131163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:58.131163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:58.131163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:58.131163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:58.131163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:59:58.131163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 14:59:58.133658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 106/212] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:58.139226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af55a7a-9a1d-4c16-bc04-1c35c2461c23 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:58.164455 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 107/213] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:59:58.175622 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:58.191737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5c800-540b-404f-bb97-92d473ed9794 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:58.247277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af55a7a-9a1d-4c16-bc04-1c35c2461c23 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:58.282888 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa281db-1d0d-4639-84cb-65d933c109e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:58.283596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa281db-1d0d-4639-84cb-65d933c109e4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:58.283879 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa281db-1d0d-4639-84cb-65d933c109e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:58.284159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa281db-1d0d-4639-84cb-65d933c109e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:58.284908 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa281db-1d0d-4639-84cb-65d933c109e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:58.285221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa281db-1d0d-4639-84cb-65d933c109e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:58.286596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa281db-1d0d-4639-84cb-65d933c109e4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:58.295431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 107/214] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 14:59:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:58.304912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa281db-1d0d-4639-84cb-65d933c109e4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:58.311743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5c800-540b-404f-bb97-92d473ed9794 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:58.332793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:58.356364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c2c26-44ac-4734-b43d-11ed463a28c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:58.358294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c2c26-44ac-4734-b43d-11ed463a28c4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:58.359077 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c2c26-44ac-4734-b43d-11ed463a28c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:58.359721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c2c26-44ac-4734-b43d-11ed463a28c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:58.361387 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c2c26-44ac-4734-b43d-11ed463a28c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:58.363135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c2c26-44ac-4734-b43d-11ed463a28c4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:58.369160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 108/215] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 14:59:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:58.372581 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c2c26-44ac-4734-b43d-11ed463a28c4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:58.404982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:58.415217 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa281db-1d0d-4639-84cb-65d933c109e4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:58.452464 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb339a-dc6e-4ab7-bdc6-a0029fa54ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:58.453596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb339a-dc6e-4ab7-bdc6-a0029fa54ebe None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:58.453760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb339a-dc6e-4ab7-bdc6-a0029fa54ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:58.454102 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb339a-dc6e-4ab7-bdc6-a0029fa54ebe None admin] PATH_INFO: `/v3/users/be0785d66c6443cc8764339d1842f72d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:58.468552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb339a-dc6e-4ab7-bdc6-a0029fa54ebe None admin] RBAC: Authorizing `identity:get_user(user_id=be0785d66c6443cc8764339d1842f72d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:58.470524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb339a-dc6e-4ab7-bdc6-a0029fa54ebe None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:58.472439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 108/216] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:58 2022] GET /identity/v3/users/be0785d66c6443cc8764339d1842f72d => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:58.478079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb339a-dc6e-4ab7-bdc6-a0029fa54ebe None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:58.495868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c2c26-44ac-4734-b43d-11ed463a28c4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:58.533133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3191f-2715-496a-89fc-7008a1d905c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:58.534261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3191f-2715-496a-89fc-7008a1d905c3 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:58.534694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3191f-2715-496a-89fc-7008a1d905c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:58.535113 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3191f-2715-496a-89fc-7008a1d905c3 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:58.535995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3191f-2715-496a-89fc-7008a1d905c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:58.538513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3191f-2715-496a-89fc-7008a1d905c3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:58.586607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb339a-dc6e-4ab7-bdc6-a0029fa54ebe None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:58.623471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48f062-4f81-4d75-bfb0-7b512854442d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:58.624132 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48f062-4f81-4d75-bfb0-7b512854442d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:58.624417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48f062-4f81-4d75-bfb0-7b512854442d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:58.624745 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48f062-4f81-4d75-bfb0-7b512854442d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:58.641692 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 109/217] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 14:59:58 2022] POST /identity/v3/users => generated 297 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:59:58.652046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:58.713886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48f062-4f81-4d75-bfb0-7b512854442d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:58.715802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48f062-4f81-4d75-bfb0-7b512854442d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:58.768858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdd894-0234-40be-876b-2707d1bd8473 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:58.769282 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdd894-0234-40be-876b-2707d1bd8473 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:58.769385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdd894-0234-40be-876b-2707d1bd8473 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:58.771102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 110/218] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:59:58.777446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a411b9-7448-437c-a886-345b6b0587e9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:58.777736 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a411b9-7448-437c-a886-345b6b0587e9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:58.778004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a411b9-7448-437c-a886-345b6b0587e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:58.796636 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-88a411b9-7448-437c-a886-345b6b0587e9 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:58.797837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-88a411b9-7448-437c-a886-345b6b0587e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:58.838354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 109/219] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 14:59:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:59:58.843874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48f062-4f81-4d75-bfb0-7b512854442d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:58.856223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:58.891335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 111/220] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:59:58.903237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:58.914549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a411b9-7448-437c-a886-345b6b0587e9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:58.968805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48f062-4f81-4d75-bfb0-7b512854442d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:59.012787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c814f5c-beac-4f93-b347-f4b5e05dc5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:59.014060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c814f5c-beac-4f93-b347-f4b5e05dc5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:59.014582 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c814f5c-beac-4f93-b347-f4b5e05dc5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:59.015055 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c814f5c-beac-4f93-b347-f4b5e05dc5c1 None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:59.016500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c814f5c-beac-4f93-b347-f4b5e05dc5c1 None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:59.018136 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c814f5c-beac-4f93-b347-f4b5e05dc5c1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:59.021144 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 110/221] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:58 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:59.027999 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c814f5c-beac-4f93-b347-f4b5e05dc5c1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:59.048440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a411b9-7448-437c-a886-345b6b0587e9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:59.103221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061b09f-56b2-4ecf-bb46-3754c51d79f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:59.103221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061b09f-56b2-4ecf-bb46-3754c51d79f7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:59.103221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061b09f-56b2-4ecf-bb46-3754c51d79f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:59.103221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061b09f-56b2-4ecf-bb46-3754c51d79f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:59.110326 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061b09f-56b2-4ecf-bb46-3754c51d79f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4061b09f-56b2-4ecf-bb46-3754c51d79f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:59.118632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:59.119595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:59.120824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:59.120824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:59.120824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:59.120824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:59.120824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:59.120824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:59.120824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:59.120824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:59.120824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:59.120824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:59.120824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:59:59.120824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 14:59:59.120824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 112/222] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 14:59:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:59:59.126293 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061b09f-56b2-4ecf-bb46-3754c51d79f7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:59.163099 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c814f5c-beac-4f93-b347-f4b5e05dc5c1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:59.163947 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:59.184430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:59.208467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a2d12-bf4c-43b5-8884-2a6feef1b0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:59.210324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a2d12-bf4c-43b5-8884-2a6feef1b0df None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:59.210614 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a2d12-bf4c-43b5-8884-2a6feef1b0df None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:59.210894 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a2d12-bf4c-43b5-8884-2a6feef1b0df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:59.211766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a2d12-bf4c-43b5-8884-2a6feef1b0df None admin] RBAC: Adding query filter params (role.id=b6dd71d4043d43e9823ec25176863347, scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487, user.id=be0785d66c6443cc8764339d1842f72d) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:59.212465 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a2d12-bf4c-43b5-8884-2a6feef1b0df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:59.214280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a2d12-bf4c-43b5-8884-2a6feef1b0df None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:59.220656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 111/223] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 14:59:59 2022] GET /identity/v3/role_assignments?user.id=be0785d66c6443cc8764339d1842f72d&scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487&role.id=b6dd71d4043d43e9823ec25176863347 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:59.221770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7483fa-da9e-4fc4-9d4f-16ee3841d1ad None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:59.222051 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7483fa-da9e-4fc4-9d4f-16ee3841d1ad None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:59.222377 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7483fa-da9e-4fc4-9d4f-16ee3841d1ad None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:59.229186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 112/224] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 14:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:59:59.232415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105c58b-c9ce-4d79-8bc5-4f0dc69ecd0d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:59.232693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105c58b-c9ce-4d79-8bc5-4f0dc69ecd0d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:59.232975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105c58b-c9ce-4d79-8bc5-4f0dc69ecd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:59.259647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061b09f-56b2-4ecf-bb46-3754c51d79f7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:59.275643 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d105c58b-c9ce-4d79-8bc5-4f0dc69ecd0d None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:59.276895 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d105c58b-c9ce-4d79-8bc5-4f0dc69ecd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:59.335008 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844e043-5179-4ef8-84c4-5dd59e180936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:59.335725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844e043-5179-4ef8-84c4-5dd59e180936 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:59.336010 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844e043-5179-4ef8-84c4-5dd59e180936 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:59.336289 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844e043-5179-4ef8-84c4-5dd59e180936 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:59.337065 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844e043-5179-4ef8-84c4-5dd59e180936 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:59.337376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844e043-5179-4ef8-84c4-5dd59e180936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:59.338805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844e043-5179-4ef8-84c4-5dd59e180936 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:59.361634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 113/225] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 14:59:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:59.367287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844e043-5179-4ef8-84c4-5dd59e180936 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:59.399845 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 113/226] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 14:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:59:59.422959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105c58b-c9ce-4d79-8bc5-4f0dc69ecd0d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:59.503157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:59.517004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844e043-5179-4ef8-84c4-5dd59e180936 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:59.562361 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b67e81-7b5c-4d36-9d8a-c3a99b63c0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:59.563277 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b67e81-7b5c-4d36-9d8a-c3a99b63c0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:59.563566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b67e81-7b5c-4d36-9d8a-c3a99b63c0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:59.563842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b67e81-7b5c-4d36-9d8a-c3a99b63c0e2 None admin] PATH_INFO: `/v3/users/cf20004de8ff441cb52689a8d2b9e0d8` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:59.566640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b67e81-7b5c-4d36-9d8a-c3a99b63c0e2 None admin] RBAC: Authorizing `identity:get_user(user_id=cf20004de8ff441cb52689a8d2b9e0d8)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:59.568124 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b67e81-7b5c-4d36-9d8a-c3a99b63c0e2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:59.574478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 114/227] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:59 2022] GET /identity/v3/users/cf20004de8ff441cb52689a8d2b9e0d8 => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:59:59.580187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b67e81-7b5c-4d36-9d8a-c3a99b63c0e2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:59.606301 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105c58b-c9ce-4d79-8bc5-4f0dc69ecd0d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:59.619238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:59.645344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9919851-3048-43b8-87bc-643ea01c73f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:59.646218 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9919851-3048-43b8-87bc-643ea01c73f1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:59.646505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9919851-3048-43b8-87bc-643ea01c73f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:59.646815 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9919851-3048-43b8-87bc-643ea01c73f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:59.658098 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9919851-3048-43b8-87bc-643ea01c73f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-e9919851-3048-43b8-87bc-643ea01c73f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:59:59.667951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:59:59.668862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:59:59.669673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:59:59.669673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:59:59.669673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:59:59.669673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:59:59.669673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:59:59.669673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:59:59.669673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:59:59.669673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:59:59.669673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:59:59.669673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:59:59.669673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:59:59.669673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 14:59:59.669673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 114/228] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 14:59:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:59:59.675271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9919851-3048-43b8-87bc-643ea01c73f1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:59.819898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9919851-3048-43b8-87bc-643ea01c73f1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:59.824811 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b67e81-7b5c-4d36-9d8a-c3a99b63c0e2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:59.859801 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882e88e-4618-40ed-a006-c7c09fb13c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:59.861157 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882e88e-4618-40ed-a006-c7c09fb13c23 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:59.861636 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882e88e-4618-40ed-a006-c7c09fb13c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:59.862108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882e88e-4618-40ed-a006-c7c09fb13c23 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:59.863070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882e88e-4618-40ed-a006-c7c09fb13c23 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:59:59.863613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882e88e-4618-40ed-a006-c7c09fb13c23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:59.865497 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882e88e-4618-40ed-a006-c7c09fb13c23 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:59.877113 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 115/229] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 14:59:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:59.882912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882e88e-4618-40ed-a006-c7c09fb13c23 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:59.887765 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75dc405-76df-411e-bb10-6361b7f70efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:59.889201 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dc405-76df-411e-bb10-6361b7f70efc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:59.889874 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dc405-76df-411e-bb10-6361b7f70efc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:59.890352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dc405-76df-411e-bb10-6361b7f70efc None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:59.891791 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75dc405-76df-411e-bb10-6361b7f70efc None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:59.893453 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75dc405-76df-411e-bb10-6361b7f70efc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:59.896342 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 115/230] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 14:59:59 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:59:59.904606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75dc405-76df-411e-bb10-6361b7f70efc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:59.937380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:59.980582 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:00.013215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882e88e-4618-40ed-a006-c7c09fb13c23 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:00.049572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381812b-41d0-419a-94e6-889fb3220299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:00.050450 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381812b-41d0-419a-94e6-889fb3220299 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:00.050737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381812b-41d0-419a-94e6-889fb3220299 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:00.051019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381812b-41d0-419a-94e6-889fb3220299 None admin] PATH_INFO: `/v3/users/cf20004de8ff441cb52689a8d2b9e0d8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:00.052288 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381812b-41d0-419a-94e6-889fb3220299 None admin] RBAC: Authorizing `identity:get_user(user_id=cf20004de8ff441cb52689a8d2b9e0d8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:00.054158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381812b-41d0-419a-94e6-889fb3220299 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:00.056032 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 116/231] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 14:59:59 2022] GET /identity/v3/users/cf20004de8ff441cb52689a8d2b9e0d8 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:00.061470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381812b-41d0-419a-94e6-889fb3220299 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:00.135408 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75dc405-76df-411e-bb10-6361b7f70efc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:00.168415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381812b-41d0-419a-94e6-889fb3220299 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:00.171575 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:00.207661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d79c34-8f24-4ed6-97cd-1a58ce18e4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:00.208632 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d79c34-8f24-4ed6-97cd-1a58ce18e4cd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:00.209028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d79c34-8f24-4ed6-97cd-1a58ce18e4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:00.209440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d79c34-8f24-4ed6-97cd-1a58ce18e4cd None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:00.211388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d79c34-8f24-4ed6-97cd-1a58ce18e4cd None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:00.212919 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d79c34-8f24-4ed6-97cd-1a58ce18e4cd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:00.215024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 117/232] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:00 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:00.220627 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d79c34-8f24-4ed6-97cd-1a58ce18e4cd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:00.228452 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1c20b-3468-4bd0-a2d0-c8395de52a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:00.229371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1c20b-3468-4bd0-a2d0-c8395de52a8b None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:00.229653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1c20b-3468-4bd0-a2d0-c8395de52a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:00.229930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1c20b-3468-4bd0-a2d0-c8395de52a8b None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487/users/cf20004de8ff441cb52689a8d2b9e0d8/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:00.236476 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1c20b-3468-4bd0-a2d0-c8395de52a8b None admin] RBAC: Authorizing `identity:create_grant(project_id=01d2a11c551c4d8fbf8eb173cd1cc487, user_id=cf20004de8ff441cb52689a8d2b9e0d8, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:00.237925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1c20b-3468-4bd0-a2d0-c8395de52a8b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:00.329079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d79c34-8f24-4ed6-97cd-1a58ce18e4cd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:00.337676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 116/233] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 14:59:59 2022] PUT /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487/users/cf20004de8ff441cb52689a8d2b9e0d8/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:00:00.368064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e040199-f088-45b8-9b98-bd1a02d3170f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:00.368995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e040199-f088-45b8-9b98-bd1a02d3170f None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:00.369344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e040199-f088-45b8-9b98-bd1a02d3170f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:00.369626 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e040199-f088-45b8-9b98-bd1a02d3170f None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487/users/cf20004de8ff441cb52689a8d2b9e0d8/roles/c077ebdf7f214d7585dfc1570eecf882` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:00.372142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e040199-f088-45b8-9b98-bd1a02d3170f None admin] RBAC: Authorizing `identity:create_grant(project_id=01d2a11c551c4d8fbf8eb173cd1cc487, user_id=cf20004de8ff441cb52689a8d2b9e0d8, role_id=c077ebdf7f214d7585dfc1570eecf882)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:00.373617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e040199-f088-45b8-9b98-bd1a02d3170f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:00.453386 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 118/234] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:00:00 2022] PUT /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487/users/cf20004de8ff441cb52689a8d2b9e0d8/roles/c077ebdf7f214d7585dfc1570eecf882 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:00:01.308623 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9a7ec-2f17-46a9-8d19-bb04c8a78c75 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:01.309339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9a7ec-2f17-46a9-8d19-bb04c8a78c75 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:01.309339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9a7ec-2f17-46a9-8d19-bb04c8a78c75 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:01.310838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 117/235] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:01.315819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bcc75-79c6-478e-a1ab-73442924e655 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:01.316639 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bcc75-79c6-478e-a1ab-73442924e655 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:01.317112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bcc75-79c6-478e-a1ab-73442924e655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:01.334068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-db4bcc75-79c6-478e-a1ab-73442924e655 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:01.336447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-db4bcc75-79c6-478e-a1ab-73442924e655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:01.361621 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:01.398861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff0cd3-abfb-4b4d-8962-489a60d6592c None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:01.399496 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff0cd3-abfb-4b4d-8962-489a60d6592c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:01.399937 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff0cd3-abfb-4b4d-8962-489a60d6592c None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:01.401612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 118/236] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:01.406796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1079983-8a2f-4c44-816d-9c41b1f30498 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:01.407370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1079983-8a2f-4c44-816d-9c41b1f30498 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:01.407851 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1079983-8a2f-4c44-816d-9c41b1f30498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:01.425003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b1079983-8a2f-4c44-816d-9c41b1f30498 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:01.426472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b1079983-8a2f-4c44-816d-9c41b1f30498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:01.429718 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 119/237] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:01.470610 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bcc75-79c6-478e-a1ab-73442924e655 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:01.503491 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 119/238] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 24 15:00:01.520890 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1079983-8a2f-4c44-816d-9c41b1f30498 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:01.540181 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:01.626347 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1079983-8a2f-4c44-816d-9c41b1f30498 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:01.665378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e6c6b-b077-4e76-bb20-b9056983f6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:01.666561 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e6c6b-b077-4e76-bb20-b9056983f6be None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:01.667043 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e6c6b-b077-4e76-bb20-b9056983f6be None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:01.667618 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e6c6b-b077-4e76-bb20-b9056983f6be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:01.680041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e6c6b-b077-4e76-bb20-b9056983f6be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-5c2e6c6b-b077-4e76-bb20-b9056983f6be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:01.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:01.689810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:01.690628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:01.690628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:01.690628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:01.690628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:01.690628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:01.690628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:01.690628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:01.690628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:01.690628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:01.690628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:01.690628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 15:00:01.690628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:01.692694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 120/239] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:01.700118 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e6c6b-b077-4e76-bb20-b9056983f6be None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:01.703972 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bcc75-79c6-478e-a1ab-73442924e655 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:01.761865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44014810-789c-4933-a466-cc1b1ec4a428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:01.762674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44014810-789c-4933-a466-cc1b1ec4a428 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:01.762961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44014810-789c-4933-a466-cc1b1ec4a428 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:01.763263 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44014810-789c-4933-a466-cc1b1ec4a428 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:01.765123 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44014810-789c-4933-a466-cc1b1ec4a428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:01.766596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44014810-789c-4933-a466-cc1b1ec4a428 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:01.773672 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 120/240] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:01.776265 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44014810-789c-4933-a466-cc1b1ec4a428 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:01.811474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:01.820015 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:01.836264 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e6c6b-b077-4e76-bb20-b9056983f6be None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:01.875234 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a825e94-1bf1-4600-84d5-357c74ca0e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:01.875919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a825e94-1bf1-4600-84d5-357c74ca0e55 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:01.876207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a825e94-1bf1-4600-84d5-357c74ca0e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:01.876540 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a825e94-1bf1-4600-84d5-357c74ca0e55 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:01.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a825e94-1bf1-4600-84d5-357c74ca0e55 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:01.877746 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a825e94-1bf1-4600-84d5-357c74ca0e55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:01.879338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a825e94-1bf1-4600-84d5-357c74ca0e55 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:01.889135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 121/241] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:01.890557 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82adf7-1f94-4447-9b12-49ddc8db60fe None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:01.890852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82adf7-1f94-4447-9b12-49ddc8db60fe None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:01.891134 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82adf7-1f94-4447-9b12-49ddc8db60fe None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:01.892839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 122/242] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:01.900178 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82adf7-1f94-4447-9b12-49ddc8db60fe None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:01.917654 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44014810-789c-4933-a466-cc1b1ec4a428 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:01.955490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f785bbf-d088-4cd1-91ad-8f20c08e0293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:01.956545 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f785bbf-d088-4cd1-91ad-8f20c08e0293 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:01.957053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f785bbf-d088-4cd1-91ad-8f20c08e0293 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:01.957523 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f785bbf-d088-4cd1-91ad-8f20c08e0293 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:01.958531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f785bbf-d088-4cd1-91ad-8f20c08e0293 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:01.960412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f785bbf-d088-4cd1-91ad-8f20c08e0293 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:02.032949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 121/243] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 15:00:01 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:00:02.033565 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82adf7-1f94-4447-9b12-49ddc8db60fe None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:02.035098 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26407109-5dde-48ff-b617-5b6f2693f4fd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:02.035718 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26407109-5dde-48ff-b617-5b6f2693f4fd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:02.036187 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26407109-5dde-48ff-b617-5b6f2693f4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:02.075368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-26407109-5dde-48ff-b617-5b6f2693f4fd None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:02.077037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-26407109-5dde-48ff-b617-5b6f2693f4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:02.077156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6c1d9-ee5f-4f0a-a208-b33e8055de0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:02.078026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6c1d9-ee5f-4f0a-a208-b33e8055de0c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:02.078317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6c1d9-ee5f-4f0a-a208-b33e8055de0c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:02.078647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6c1d9-ee5f-4f0a-a208-b33e8055de0c None admin] PATH_INFO: `/v3/users/53a386e88ed14336a5c8c46aa95bdd74` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:02.114814 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:02.122174 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d6c1d9-ee5f-4f0a-a208-b33e8055de0c None admin] RBAC: Authorizing `identity:get_user(user_id=53a386e88ed14336a5c8c46aa95bdd74)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:02.126040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d6c1d9-ee5f-4f0a-a208-b33e8055de0c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:02.131222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 123/244] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:00:01 2022] GET /identity/v3/users/53a386e88ed14336a5c8c46aa95bdd74 => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:02.136658 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6c1d9-ee5f-4f0a-a208-b33e8055de0c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:02.199066 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 122/245] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:02.224363 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26407109-5dde-48ff-b617-5b6f2693f4fd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:02.245324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:02.311153 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6c1d9-ee5f-4f0a-a208-b33e8055de0c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:02.350381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab0266-f0ac-4bda-b7fb-a418e1383b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:02.351482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab0266-f0ac-4bda-b7fb-a418e1383b17 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:02.351961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab0266-f0ac-4bda-b7fb-a418e1383b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:02.352424 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab0266-f0ac-4bda-b7fb-a418e1383b17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:02.360351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26407109-5dde-48ff-b617-5b6f2693f4fd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:02.402764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38563b0-c908-4637-89e3-73304194ed4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:02.403970 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38563b0-c908-4637-89e3-73304194ed4f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:02.404508 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38563b0-c908-4637-89e3-73304194ed4f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:02.405121 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38563b0-c908-4637-89e3-73304194ed4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:02.416990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38563b0-c908-4637-89e3-73304194ed4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d38563b0-c908-4637-89e3-73304194ed4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:02.426061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:02.427049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:02.427902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:02.427902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:02.427902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:02.427902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:02.427902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:02.427902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:02.427902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:02.427902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:02.427902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:02.427902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:02.427902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:02.427902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:02.430987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 123/246] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:02.437047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:02.437186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38563b0-c908-4637-89e3-73304194ed4f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:02.441958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab0266-f0ac-4bda-b7fb-a418e1383b17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:02.444246 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab0266-f0ac-4bda-b7fb-a418e1383b17 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:02.532334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:02.570132 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 124/247] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 15:00:02.574834 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38563b0-c908-4637-89e3-73304194ed4f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:02.578109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab0266-f0ac-4bda-b7fb-a418e1383b17 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:02.621863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0339ac-b245-4226-8677-4b27a5ce39bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:02.622632 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0339ac-b245-4226-8677-4b27a5ce39bb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:02.622984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0339ac-b245-4226-8677-4b27a5ce39bb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:02.623358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0339ac-b245-4226-8677-4b27a5ce39bb None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:02.624198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0339ac-b245-4226-8677-4b27a5ce39bb None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:02.624562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0339ac-b245-4226-8677-4b27a5ce39bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:02.626027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0339ac-b245-4226-8677-4b27a5ce39bb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:02.636695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 124/248] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:02.643497 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0339ac-b245-4226-8677-4b27a5ce39bb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:02.701993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:02.705824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab0266-f0ac-4bda-b7fb-a418e1383b17 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:02.750712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d803408-9d38-4f0c-9dba-546b7d84c74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:02.751992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d803408-9d38-4f0c-9dba-546b7d84c74b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:02.752177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d803408-9d38-4f0c-9dba-546b7d84c74b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:02.752624 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d803408-9d38-4f0c-9dba-546b7d84c74b None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:02.753952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d803408-9d38-4f0c-9dba-546b7d84c74b None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:02.755641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d803408-9d38-4f0c-9dba-546b7d84c74b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:02.758537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 125/249] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:02 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:02.764982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d803408-9d38-4f0c-9dba-546b7d84c74b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:02.781539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0339ac-b245-4226-8677-4b27a5ce39bb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:02.827413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f551ec-485c-4c3c-93a6-9d3c90602c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:02.828461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f551ec-485c-4c3c-93a6-9d3c90602c6a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:02.828844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f551ec-485c-4c3c-93a6-9d3c90602c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:02.829223 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f551ec-485c-4c3c-93a6-9d3c90602c6a None admin] PATH_INFO: `/v3/users/be0785d66c6443cc8764339d1842f72d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:02.830614 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f551ec-485c-4c3c-93a6-9d3c90602c6a None admin] RBAC: Authorizing `identity:get_user(user_id=be0785d66c6443cc8764339d1842f72d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:02.832227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f551ec-485c-4c3c-93a6-9d3c90602c6a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:02.834215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 125/250] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:00:02 2022] GET /identity/v3/users/be0785d66c6443cc8764339d1842f72d => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:02.841294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f551ec-485c-4c3c-93a6-9d3c90602c6a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:02.844519 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:02.899203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d803408-9d38-4f0c-9dba-546b7d84c74b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:02.944751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55715028-ac4e-4796-a527-990aac81e8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:02.945785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55715028-ac4e-4796-a527-990aac81e8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:02.945936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55715028-ac4e-4796-a527-990aac81e8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:02.946324 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55715028-ac4e-4796-a527-990aac81e8a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:02.947320 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55715028-ac4e-4796-a527-990aac81e8a6 None admin] RBAC: Adding query filter params (role.id=ae37f175be764624b8ef6f3c0104dbf9, scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487, user.id=53a386e88ed14336a5c8c46aa95bdd74) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:02.947740 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55715028-ac4e-4796-a527-990aac81e8a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:02.950477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55715028-ac4e-4796-a527-990aac81e8a6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:02.957193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 126/251] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:02 2022] GET /identity/v3/role_assignments?user.id=53a386e88ed14336a5c8c46aa95bdd74&scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487&role.id=ae37f175be764624b8ef6f3c0104dbf9 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:02.966518 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a904bc-674b-433d-bcf6-796cdc9c8ac3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:02.966741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f551ec-485c-4c3c-93a6-9d3c90602c6a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:02.967089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a904bc-674b-433d-bcf6-796cdc9c8ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:02.967499 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a904bc-674b-433d-bcf6-796cdc9c8ac3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:02.969224 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 127/252] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:02.981536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ceb5c-8aa1-47a8-9e27-a1788e7f3035 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:02.981764 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ceb5c-8aa1-47a8-9e27-a1788e7f3035 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:02.982102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ceb5c-8aa1-47a8-9e27-a1788e7f3035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:03.009550 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989fcd1-d35f-4f3a-8980-7c29957b4692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:03.010682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989fcd1-d35f-4f3a-8980-7c29957b4692 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:03.010954 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989fcd1-d35f-4f3a-8980-7c29957b4692 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:03.011754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989fcd1-d35f-4f3a-8980-7c29957b4692 None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:03.012736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989fcd1-d35f-4f3a-8980-7c29957b4692 None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:03.014188 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989fcd1-d35f-4f3a-8980-7c29957b4692 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:03.016225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e48ceb5c-8aa1-47a8-9e27-a1788e7f3035 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:03.016349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 126/253] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:02 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:03.017529 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e48ceb5c-8aa1-47a8-9e27-a1788e7f3035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:03.017708 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b94715-d7e1-4ddb-8661-ad5b5f065123 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:03.017946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b94715-d7e1-4ddb-8661-ad5b5f065123 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:03.018221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b94715-d7e1-4ddb-8661-ad5b5f065123 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:03.019795 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 127/254] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:03.021354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b94715-d7e1-4ddb-8661-ad5b5f065123 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:03.051830 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:03.110972 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 128/255] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 15:00:03.113181 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83a49e-8e84-40c4-8ab4-5552f38bdc86 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:03.113714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83a49e-8e84-40c4-8ab4-5552f38bdc86 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:03.114187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83a49e-8e84-40c4-8ab4-5552f38bdc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:03.141975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:03.142440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ad83a49e-8e84-40c4-8ab4-5552f38bdc86 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:03.143701 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ad83a49e-8e84-40c4-8ab4-5552f38bdc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:03.202294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b94715-d7e1-4ddb-8661-ad5b5f065123 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:03.256849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609a67a-91d1-428e-b7c6-298f9f4f6ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:03.257811 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609a67a-91d1-428e-b7c6-298f9f4f6ca1 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:03.258137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609a67a-91d1-428e-b7c6-298f9f4f6ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:03.258422 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609a67a-91d1-428e-b7c6-298f9f4f6ca1 None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487/users/be0785d66c6443cc8764339d1842f72d/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:03.262463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609a67a-91d1-428e-b7c6-298f9f4f6ca1 None admin] RBAC: Authorizing `identity:create_grant(project_id=01d2a11c551c4d8fbf8eb173cd1cc487, user_id=be0785d66c6443cc8764339d1842f72d, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:03.266013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609a67a-91d1-428e-b7c6-298f9f4f6ca1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:03.276751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 129/256] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:03.277618 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83a49e-8e84-40c4-8ab4-5552f38bdc86 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:03.376821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 128/257] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:00:03 2022] PUT /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487/users/be0785d66c6443cc8764339d1842f72d/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:00:03.377827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609a67a-91d1-428e-b7c6-298f9f4f6ca1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:03.408899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:03.424127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83a49e-8e84-40c4-8ab4-5552f38bdc86 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:03.497775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299e68a-33e9-44b9-a247-747da654e1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:03.499133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299e68a-33e9-44b9-a247-747da654e1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:03.502751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299e68a-33e9-44b9-a247-747da654e1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:03.503211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299e68a-33e9-44b9-a247-747da654e1c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:03.531511 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299e68a-33e9-44b9-a247-747da654e1c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-c299e68a-33e9-44b9-a247-747da654e1c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:03.548172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:03.549372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:03.550386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:03.550386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:03.550386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:03.550386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:03.550386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:03.550386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:03.550386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:03.550386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:03.550386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:03.550386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:03.550386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:00:03.550386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:03.552672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 130/258] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:00:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:03.559296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299e68a-33e9-44b9-a247-747da654e1c7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:03.573923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609a67a-91d1-428e-b7c6-298f9f4f6ca1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:03.609889 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:03.619332 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e098ea3-625c-4456-86eb-747fc8a4dcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:03.620317 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e098ea3-625c-4456-86eb-747fc8a4dcfe None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:03.620608 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e098ea3-625c-4456-86eb-747fc8a4dcfe None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:03.620884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e098ea3-625c-4456-86eb-747fc8a4dcfe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:03.639215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e098ea3-625c-4456-86eb-747fc8a4dcfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-6e098ea3-625c-4456-86eb-747fc8a4dcfe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:03.644649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:03.645566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:03.647336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:03.647336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:03.647336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:03.647336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:03.647336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:03.647336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:03.647336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:03.647336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:03.647336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:03.647336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:03.647336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:03.647336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:03.647336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 129/259] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:00:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:03.652482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e098ea3-625c-4456-86eb-747fc8a4dcfe None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:03.789965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299e68a-33e9-44b9-a247-747da654e1c7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:03.803167 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e098ea3-625c-4456-86eb-747fc8a4dcfe None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:03.876836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362b6ab-93d2-4e34-81c7-860ff22e6be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:03.877521 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362b6ab-93d2-4e34-81c7-860ff22e6be8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:03.877808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362b6ab-93d2-4e34-81c7-860ff22e6be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:03.878089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362b6ab-93d2-4e34-81c7-860ff22e6be8 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:03.878844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362b6ab-93d2-4e34-81c7-860ff22e6be8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:03.879155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362b6ab-93d2-4e34-81c7-860ff22e6be8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:03.880567 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362b6ab-93d2-4e34-81c7-860ff22e6be8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:03.889255 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4900479-ea81-4158-a887-9888f0811e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:03.890126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4900479-ea81-4158-a887-9888f0811e2c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:03.890587 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4900479-ea81-4158-a887-9888f0811e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:03.891016 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4900479-ea81-4158-a887-9888f0811e2c None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:03.891946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4900479-ea81-4158-a887-9888f0811e2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:03.892402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4900479-ea81-4158-a887-9888f0811e2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:03.893661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 130/260] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:03.895508 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4900479-ea81-4158-a887-9888f0811e2c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:03.902198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362b6ab-93d2-4e34-81c7-860ff22e6be8 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:03.917406 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 131/261] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:03.935879 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4900479-ea81-4158-a887-9888f0811e2c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:03.970467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:04.044072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:04.178975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362b6ab-93d2-4e34-81c7-860ff22e6be8 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:04.237460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4900479-ea81-4158-a887-9888f0811e2c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:04.269776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77e05a-1387-4956-8707-b6c0fef6330b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:04.272302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77e05a-1387-4956-8707-b6c0fef6330b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:04.273743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77e05a-1387-4956-8707-b6c0fef6330b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:04.274668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77e05a-1387-4956-8707-b6c0fef6330b None admin] PATH_INFO: `/v3/users/cf20004de8ff441cb52689a8d2b9e0d8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:04.277440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77e05a-1387-4956-8707-b6c0fef6330b None admin] RBAC: Authorizing `identity:get_user(user_id=cf20004de8ff441cb52689a8d2b9e0d8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:04.280915 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77e05a-1387-4956-8707-b6c0fef6330b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:04.285285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 131/262] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:00:03 2022] GET /identity/v3/users/cf20004de8ff441cb52689a8d2b9e0d8 => generated 283 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:04.291392 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77e05a-1387-4956-8707-b6c0fef6330b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:04.327394 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428db96-2d81-4591-aab2-579d0a5a660e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:04.327965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428db96-2d81-4591-aab2-579d0a5a660e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:04.328354 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428db96-2d81-4591-aab2-579d0a5a660e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:04.328754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428db96-2d81-4591-aab2-579d0a5a660e None admin] PATH_INFO: `/v3/users/cf20004de8ff441cb52689a8d2b9e0d8` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:04.333049 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428db96-2d81-4591-aab2-579d0a5a660e None admin] RBAC: Authorizing `identity:get_user(user_id=cf20004de8ff441cb52689a8d2b9e0d8)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:04.335369 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428db96-2d81-4591-aab2-579d0a5a660e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:04.338232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 132/263] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:00:03 2022] GET /identity/v3/users/cf20004de8ff441cb52689a8d2b9e0d8 => generated 283 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:04.343534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428db96-2d81-4591-aab2-579d0a5a660e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:04.468339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77e05a-1387-4956-8707-b6c0fef6330b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:04.510742 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:04.536818 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38b9c5-5927-4fd5-901e-0bcd1c690561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:04.537534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38b9c5-5927-4fd5-901e-0bcd1c690561 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:04.537905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38b9c5-5927-4fd5-901e-0bcd1c690561 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:04.538192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38b9c5-5927-4fd5-901e-0bcd1c690561 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:04.557384 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:04.602997 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428db96-2d81-4591-aab2-579d0a5a660e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:04.640245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38b9c5-5927-4fd5-901e-0bcd1c690561 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:04.641709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38b9c5-5927-4fd5-901e-0bcd1c690561 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:04.700434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89821c-590c-4465-9d04-fe8f88610371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:04.701111 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89821c-590c-4465-9d04-fe8f88610371 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:04.701402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89821c-590c-4465-9d04-fe8f88610371 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:04.701688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89821c-590c-4465-9d04-fe8f88610371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:04.792759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 132/264] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:04.799196 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38b9c5-5927-4fd5-901e-0bcd1c690561 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:04.863100 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:04.891398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89821c-590c-4465-9d04-fe8f88610371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:04.892889 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89821c-590c-4465-9d04-fe8f88610371 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:05.047748 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:05.062021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38b9c5-5927-4fd5-901e-0bcd1c690561 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:05.104105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 133/265] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 761 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 15:00:05.109522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89821c-590c-4465-9d04-fe8f88610371 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:05.130693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c751fa-45b7-4f24-9906-0afddc871a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:05.131798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c751fa-45b7-4f24-9906-0afddc871a4c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:05.132182 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c751fa-45b7-4f24-9906-0afddc871a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:05.132465 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c751fa-45b7-4f24-9906-0afddc871a4c None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:05.133752 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c751fa-45b7-4f24-9906-0afddc871a4c None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:05.135360 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c751fa-45b7-4f24-9906-0afddc871a4c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:05.137083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 133/266] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:04 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:05.142828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c751fa-45b7-4f24-9906-0afddc871a4c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:05.313320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:05.328374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c751fa-45b7-4f24-9906-0afddc871a4c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:05.338842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89821c-590c-4465-9d04-fe8f88610371 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:05.409660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6905f9a-f3de-4afb-be50-fdbd50762da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:05.410334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6905f9a-f3de-4afb-be50-fdbd50762da8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:05.410617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6905f9a-f3de-4afb-be50-fdbd50762da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:05.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6905f9a-f3de-4afb-be50-fdbd50762da8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:05.411767 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6905f9a-f3de-4afb-be50-fdbd50762da8 None admin] RBAC: Adding query filter params (role.id=c077ebdf7f214d7585dfc1570eecf882, scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487, user.id=cf20004de8ff441cb52689a8d2b9e0d8) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:05.412083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6905f9a-f3de-4afb-be50-fdbd50762da8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:05.414390 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6905f9a-f3de-4afb-be50-fdbd50762da8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:05.417946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee5739-b476-46a9-91a9-f3a3ea8ac2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:05.418821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee5739-b476-46a9-91a9-f3a3ea8ac2ee None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:05.419104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee5739-b476-46a9-91a9-f3a3ea8ac2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:05.423570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee5739-b476-46a9-91a9-f3a3ea8ac2ee None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:05.425020 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee5739-b476-46a9-91a9-f3a3ea8ac2ee None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:05.426492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee5739-b476-46a9-91a9-f3a3ea8ac2ee None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:05.429092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 134/267] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:05 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:05.434035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 134/268] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:05 2022] GET /identity/v3/role_assignments?user.id=cf20004de8ff441cb52689a8d2b9e0d8&scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487&role.id=c077ebdf7f214d7585dfc1570eecf882 => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:05.439803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee5739-b476-46a9-91a9-f3a3ea8ac2ee None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:05.516940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:05.594131 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee5739-b476-46a9-91a9-f3a3ea8ac2ee None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:05.650007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42add7d-ece4-4417-8d82-9cf774847926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:05.650696 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42add7d-ece4-4417-8d82-9cf774847926 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:05.650987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42add7d-ece4-4417-8d82-9cf774847926 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:05.651302 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42add7d-ece4-4417-8d82-9cf774847926 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:05.652164 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42add7d-ece4-4417-8d82-9cf774847926 None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487, user.id=cf20004de8ff441cb52689a8d2b9e0d8) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:05.652483 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42add7d-ece4-4417-8d82-9cf774847926 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:05.653957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42add7d-ece4-4417-8d82-9cf774847926 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:05.662485 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 135/269] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:05 2022] GET /identity/v3/role_assignments?user.id=cf20004de8ff441cb52689a8d2b9e0d8&scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:05.671168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490545c-65f0-468c-8c23-dbd7cab74070 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:05.672181 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490545c-65f0-468c-8c23-dbd7cab74070 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:05.672666 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490545c-65f0-468c-8c23-dbd7cab74070 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:05.674636 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 135/270] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:05.681546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34633dc0-bd28-4673-8f79-d244534ab53a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:05.681857 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34633dc0-bd28-4673-8f79-d244534ab53a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:05.682122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34633dc0-bd28-4673-8f79-d244534ab53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:05.707499 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-34633dc0-bd28-4673-8f79-d244534ab53a None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:05.708734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-34633dc0-bd28-4673-8f79-d244534ab53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:05.810126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 136/271] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:05.850536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490545c-65f0-468c-8c23-dbd7cab74070 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:05.870762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfc8d1-d120-4ae0-8ab6-88487c3123bc None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:05.871157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfc8d1-d120-4ae0-8ab6-88487c3123bc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:05.871465 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfc8d1-d120-4ae0-8ab6-88487c3123bc None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:05.873148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 137/272] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:05.875936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b31d8-77db-44ba-8f1b-899214ef1b76 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:05.876228 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b31d8-77db-44ba-8f1b-899214ef1b76 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:05.876509 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b31d8-77db-44ba-8f1b-899214ef1b76 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:05.878130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 138/273] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:05.879867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0981d-303d-4a1a-abc4-a40d479a5c0d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:05.880367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0981d-303d-4a1a-abc4-a40d479a5c0d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:05.880925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0981d-303d-4a1a-abc4-a40d479a5c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:05.899008 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-afb0981d-303d-4a1a-abc4-a40d479a5c0d None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:05.900225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-afb0981d-303d-4a1a-abc4-a40d479a5c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:05.960032 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:05.986990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490545c-65f0-468c-8c23-dbd7cab74070 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:06.002272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 139/274] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:06.003616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f7ebf-0bac-46a9-8d8a-98d0967245eb None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:06.003904 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f7ebf-0bac-46a9-8d8a-98d0967245eb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:06.004190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f7ebf-0bac-46a9-8d8a-98d0967245eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:06.024670 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-167f7ebf-0bac-46a9-8d8a-98d0967245eb None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:06.026144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-167f7ebf-0bac-46a9-8d8a-98d0967245eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:06.040770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948c021-315a-4742-b20e-eddbe573687a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:06.042092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948c021-315a-4742-b20e-eddbe573687a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:06.042636 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948c021-315a-4742-b20e-eddbe573687a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:06.042965 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948c021-315a-4742-b20e-eddbe573687a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:06.056101 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948c021-315a-4742-b20e-eddbe573687a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-3948c021-315a-4742-b20e-eddbe573687a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:06.064878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:06.065783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:06.066633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:06.066633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:06.066633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:06.066633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:06.066633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:06.066633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:06.066633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:06.066633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:06.066633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:06.066633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:06.066633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:06.066633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:06.067849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 136/275] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:06.067849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948c021-315a-4742-b20e-eddbe573687a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:06.084771 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:06.106928 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 140/276] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:06.107640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f7ebf-0bac-46a9-8d8a-98d0967245eb None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:06.174950 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948c021-315a-4742-b20e-eddbe573687a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:06.213105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278d910-72f5-43e5-b480-c14beda81911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:06.213826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278d910-72f5-43e5-b480-c14beda81911 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:06.214113 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278d910-72f5-43e5-b480-c14beda81911 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:06.214393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278d910-72f5-43e5-b480-c14beda81911 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:06.215673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278d910-72f5-43e5-b480-c14beda81911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:06.217130 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278d910-72f5-43e5-b480-c14beda81911 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:06.218755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 137/277] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:06.219543 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278d910-72f5-43e5-b480-c14beda81911 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:06.228017 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f7ebf-0bac-46a9-8d8a-98d0967245eb None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:06.230406 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:06.269362 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7b796-312c-43f8-93f8-45d078d0bf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:06.270033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7b796-312c-43f8-93f8-45d078d0bf5a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:06.270323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7b796-312c-43f8-93f8-45d078d0bf5a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:06.270599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7b796-312c-43f8-93f8-45d078d0bf5a None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:06.271383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7b796-312c-43f8-93f8-45d078d0bf5a None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:06.272246 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7b796-312c-43f8-93f8-45d078d0bf5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:06.273869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7b796-312c-43f8-93f8-45d078d0bf5a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:06.281971 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 141/278] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:06.282852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7b796-312c-43f8-93f8-45d078d0bf5a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:06.312423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:06.331149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278d910-72f5-43e5-b480-c14beda81911 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:06.371475 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589db8b8-d2b9-46dd-9f9b-cd08fc098cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:06.372510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589db8b8-d2b9-46dd-9f9b-cd08fc098cfb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:06.372999 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589db8b8-d2b9-46dd-9f9b-cd08fc098cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:06.373277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589db8b8-d2b9-46dd-9f9b-cd08fc098cfb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:06.384214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589db8b8-d2b9-46dd-9f9b-cd08fc098cfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:06.389902 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7b796-312c-43f8-93f8-45d078d0bf5a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-589db8b8-d2b9-46dd-9f9b-cd08fc098cfb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:06.391976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:06.393092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:06.393922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:06.393922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:06.393922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:06.393922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:06.393922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:06.393922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:06.393922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:06.393922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:06.393922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:06.393922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:06.393922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 15:00:06.393922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:06.393922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 138/279] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:06.393922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589db8b8-d2b9-46dd-9f9b-cd08fc098cfb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:06.432309 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b200b-f7dd-4839-95ac-43b4d89332fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:06.432982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b200b-f7dd-4839-95ac-43b4d89332fd None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:06.433263 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b200b-f7dd-4839-95ac-43b4d89332fd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:06.433533 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b200b-f7dd-4839-95ac-43b4d89332fd None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:06.434231 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b200b-f7dd-4839-95ac-43b4d89332fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:06.435699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b200b-f7dd-4839-95ac-43b4d89332fd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:06.502782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589db8b8-d2b9-46dd-9f9b-cd08fc098cfb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:06.514315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 142/280] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 15:00:06 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:00:06.514960 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b200b-f7dd-4839-95ac-43b4d89332fd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:06.540628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:06.556109 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8b21b-b124-4d99-8385-f5b7e79f2b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:06.557058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8b21b-b124-4d99-8385-f5b7e79f2b81 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:06.557442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8b21b-b124-4d99-8385-f5b7e79f2b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:06.557635 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8b21b-b124-4d99-8385-f5b7e79f2b81 None admin] PATH_INFO: `/v3/users/6edee7cc751f41d6a9dff0779f260069` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:06.570714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:06.574649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8b21b-b124-4d99-8385-f5b7e79f2b81 None admin] RBAC: Authorizing `identity:get_user(user_id=6edee7cc751f41d6a9dff0779f260069)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:06.576270 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8b21b-b124-4d99-8385-f5b7e79f2b81 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:06.578146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 139/281] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:00:06 2022] GET /identity/v3/users/6edee7cc751f41d6a9dff0779f260069 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:06.579551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b323a0-f5ac-46d0-a1b7-e4db654855e4 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:06.579839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b323a0-f5ac-46d0-a1b7-e4db654855e4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:06.580114 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b323a0-f5ac-46d0-a1b7-e4db654855e4 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:06.581638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 140/282] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:06.584194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b323a0-f5ac-46d0-a1b7-e4db654855e4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:06.644913 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b200b-f7dd-4839-95ac-43b4d89332fd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:06.688103 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ec3f4-a0da-4f51-8649-06eb9e9ea802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:06.688562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ec3f4-a0da-4f51-8649-06eb9e9ea802 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:06.688851 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ec3f4-a0da-4f51-8649-06eb9e9ea802 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:06.689129 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ec3f4-a0da-4f51-8649-06eb9e9ea802 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:06.689895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ec3f4-a0da-4f51-8649-06eb9e9ea802 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:06.690195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ec3f4-a0da-4f51-8649-06eb9e9ea802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:06.692417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ec3f4-a0da-4f51-8649-06eb9e9ea802 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:06.700453 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b323a0-f5ac-46d0-a1b7-e4db654855e4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:06.701609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 143/283] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:06.702851 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390991a-d79d-400d-a44b-8805a52cac15 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:06.703140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390991a-d79d-400d-a44b-8805a52cac15 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:06.703452 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390991a-d79d-400d-a44b-8805a52cac15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:06.721195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0390991a-d79d-400d-a44b-8805a52cac15 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:06.722403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0390991a-d79d-400d-a44b-8805a52cac15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:06.744072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf21cb3-0f5a-462c-9218-5e2081aefb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:06.744741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf21cb3-0f5a-462c-9218-5e2081aefb0b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:06.745026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf21cb3-0f5a-462c-9218-5e2081aefb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:06.745301 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf21cb3-0f5a-462c-9218-5e2081aefb0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:06.746318 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf21cb3-0f5a-462c-9218-5e2081aefb0b None admin] RBAC: Adding query filter params (role.id=b6dd71d4043d43e9823ec25176863347, user.id=6edee7cc751f41d6a9dff0779f260069, scope.system=all) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:06.746431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf21cb3-0f5a-462c-9218-5e2081aefb0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:06.747964 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf21cb3-0f5a-462c-9218-5e2081aefb0b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:06.754457 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 141/284] 10.222.0.67 () {56 vars in 1308 bytes} [Fri Jun 24 15:00:06 2022] GET /identity/v3/role_assignments?user.id=6edee7cc751f41d6a9dff0779f260069&scope.system=all&role.id=b6dd71d4043d43e9823ec25176863347 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:06.755058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf21cb3-0f5a-462c-9218-5e2081aefb0b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:06.839694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 144/285] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:06.865023 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390991a-d79d-400d-a44b-8805a52cac15 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:06.866530 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:06.886518 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:06.924906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf21cb3-0f5a-462c-9218-5e2081aefb0b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:06.969343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632f9b6-63e2-475b-a149-b9bddbf603bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:06.970230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632f9b6-63e2-475b-a149-b9bddbf603bd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:06.970517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632f9b6-63e2-475b-a149-b9bddbf603bd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:06.970802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632f9b6-63e2-475b-a149-b9bddbf603bd None admin] PATH_INFO: `/v3/users/53a386e88ed14336a5c8c46aa95bdd74` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:06.972107 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632f9b6-63e2-475b-a149-b9bddbf603bd None admin] RBAC: Authorizing `identity:get_user(user_id=53a386e88ed14336a5c8c46aa95bdd74)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:06.973563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632f9b6-63e2-475b-a149-b9bddbf603bd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:06.976242 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 142/286] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:00:06 2022] GET /identity/v3/users/53a386e88ed14336a5c8c46aa95bdd74 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:06.981717 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632f9b6-63e2-475b-a149-b9bddbf603bd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:06.993118 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390991a-d79d-400d-a44b-8805a52cac15 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:07.035651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ecc70-a92a-4e2e-86e5-3b4f95ba79eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:07.036655 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ecc70-a92a-4e2e-86e5-3b4f95ba79eb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:07.036942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ecc70-a92a-4e2e-86e5-3b4f95ba79eb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:07.037924 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ecc70-a92a-4e2e-86e5-3b4f95ba79eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:07.050167 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ecc70-a92a-4e2e-86e5-3b4f95ba79eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f83ecc70-a92a-4e2e-86e5-3b4f95ba79eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:07.060185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:07.061105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:07.061894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:07.061894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:07.061894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:07.061894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:07.061894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:07.061894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:07.061894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:07.061894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:07.061894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:07.061894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:07.061894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:07.061894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:07.061894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 145/287] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:07.067973 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ecc70-a92a-4e2e-86e5-3b4f95ba79eb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:07.161094 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632f9b6-63e2-475b-a149-b9bddbf603bd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:07.192562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:07.200246 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42106eca-6d19-4965-be4b-9e8ff4f78129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:07.201219 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42106eca-6d19-4965-be4b-9e8ff4f78129 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:07.201453 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42106eca-6d19-4965-be4b-9e8ff4f78129 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:07.201735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42106eca-6d19-4965-be4b-9e8ff4f78129 None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:07.202938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42106eca-6d19-4965-be4b-9e8ff4f78129 None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:07.207203 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42106eca-6d19-4965-be4b-9e8ff4f78129 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:07.207203 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 143/288] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:06 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:07.220155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42106eca-6d19-4965-be4b-9e8ff4f78129 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:07.223525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:07.256625 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ecc70-a92a-4e2e-86e5-3b4f95ba79eb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:07.295679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf77896-dc7d-4863-b680-7ad29148b608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:07.296687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf77896-dc7d-4863-b680-7ad29148b608 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:07.297257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf77896-dc7d-4863-b680-7ad29148b608 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:07.297728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf77896-dc7d-4863-b680-7ad29148b608 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:07.298663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf77896-dc7d-4863-b680-7ad29148b608 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:07.299160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf77896-dc7d-4863-b680-7ad29148b608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:07.300706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf77896-dc7d-4863-b680-7ad29148b608 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:07.315100 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 146/289] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:07.320279 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf77896-dc7d-4863-b680-7ad29148b608 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:07.392430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42106eca-6d19-4965-be4b-9e8ff4f78129 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:07.433687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cac49e-e807-4f5b-aae4-f77671098bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:07.435143 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cac49e-e807-4f5b-aae4-f77671098bb2 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:07.435837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cac49e-e807-4f5b-aae4-f77671098bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:07.436308 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cac49e-e807-4f5b-aae4-f77671098bb2 None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487/users/53a386e88ed14336a5c8c46aa95bdd74/roles/ae37f175be764624b8ef6f3c0104dbf9` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:07.439133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cac49e-e807-4f5b-aae4-f77671098bb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=01d2a11c551c4d8fbf8eb173cd1cc487, user_id=53a386e88ed14336a5c8c46aa95bdd74, role_id=ae37f175be764624b8ef6f3c0104dbf9)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:07.440951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cac49e-e807-4f5b-aae4-f77671098bb2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:07.482904 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf77896-dc7d-4863-b680-7ad29148b608 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:07.518511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 144/290] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:00:07 2022] PUT /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487/users/53a386e88ed14336a5c8c46aa95bdd74/roles/ae37f175be764624b8ef6f3c0104dbf9 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:00:07.526457 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd4a8e-ccd0-4e53-85f0-a944994d43cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:07.527753 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd4a8e-ccd0-4e53-85f0-a944994d43cd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:07.528237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd4a8e-ccd0-4e53-85f0-a944994d43cd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:07.528789 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd4a8e-ccd0-4e53-85f0-a944994d43cd None admin] PATH_INFO: `/v3/users/be0785d66c6443cc8764339d1842f72d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:07.530356 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd4a8e-ccd0-4e53-85f0-a944994d43cd None admin] RBAC: Authorizing `identity:get_user(user_id=be0785d66c6443cc8764339d1842f72d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:07.532110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd4a8e-ccd0-4e53-85f0-a944994d43cd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:07.534052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 147/291] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:00:07 2022] GET /identity/v3/users/be0785d66c6443cc8764339d1842f72d => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:07.540541 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cac49e-e807-4f5b-aae4-f77671098bb2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:07.625961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:07.681443 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cac49e-e807-4f5b-aae4-f77671098bb2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:07.722003 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb974c-4e37-44b6-9d51-4b528b49164f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:07.722753 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb974c-4e37-44b6-9d51-4b528b49164f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:07.722921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb974c-4e37-44b6-9d51-4b528b49164f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:07.723215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb974c-4e37-44b6-9d51-4b528b49164f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:07.799547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb974c-4e37-44b6-9d51-4b528b49164f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:07.801002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb974c-4e37-44b6-9d51-4b528b49164f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:07.875728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:07.937154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 145/292] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:07.942419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd4a8e-ccd0-4e53-85f0-a944994d43cd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:07.945933 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:08.051417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd4a8e-ccd0-4e53-85f0-a944994d43cd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:08.096109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accf765-0ec7-4cc4-9f47-db4fbfd82c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:08.097023 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accf765-0ec7-4cc4-9f47-db4fbfd82c3d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:08.097310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accf765-0ec7-4cc4-9f47-db4fbfd82c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:08.097588 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accf765-0ec7-4cc4-9f47-db4fbfd82c3d None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:08.100594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2accf765-0ec7-4cc4-9f47-db4fbfd82c3d None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:08.102257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2accf765-0ec7-4cc4-9f47-db4fbfd82c3d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:08.104537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 148/293] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:07 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:08.112352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb974c-4e37-44b6-9d51-4b528b49164f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:08.240130 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb974c-4e37-44b6-9d51-4b528b49164f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:08.294639 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f8c20-98c5-4ff6-acfb-85b3269a83e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:08.295404 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f8c20-98c5-4ff6-acfb-85b3269a83e1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:08.295696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f8c20-98c5-4ff6-acfb-85b3269a83e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:08.295980 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f8c20-98c5-4ff6-acfb-85b3269a83e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:08.296811 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f8c20-98c5-4ff6-acfb-85b3269a83e1 None admin] RBAC: Adding query filter params (role.id=b6dd71d4043d43e9823ec25176863347, scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487, user.id=be0785d66c6443cc8764339d1842f72d) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:08.297130 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f8c20-98c5-4ff6-acfb-85b3269a83e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:08.298588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f8c20-98c5-4ff6-acfb-85b3269a83e1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:08.309157 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 146/294] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:08 2022] GET /identity/v3/role_assignments?user.id=be0785d66c6443cc8764339d1842f72d&scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487&role.id=b6dd71d4043d43e9823ec25176863347 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:09.039518 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84704de0-f05e-403a-aaf4-9454d8ad6e85 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:09.040064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84704de0-f05e-403a-aaf4-9454d8ad6e85 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:09.040064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84704de0-f05e-403a-aaf4-9454d8ad6e85 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:09.041944 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 149/295] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:09.047242 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a46153-81c9-42e0-b5e4-07499a092d81 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:09.047654 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a46153-81c9-42e0-b5e4-07499a092d81 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:09.047958 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a46153-81c9-42e0-b5e4-07499a092d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:09.066848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a3a46153-81c9-42e0-b5e4-07499a092d81 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:09.068070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a3a46153-81c9-42e0-b5e4-07499a092d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:09.089038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:09.146533 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 147/296] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:09.171629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84704de0-f05e-403a-aaf4-9454d8ad6e85 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:09.185732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f99ce9-9275-44bb-bd15-38f03fcbb017 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:09.186272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f99ce9-9275-44bb-bd15-38f03fcbb017 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:09.186993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f99ce9-9275-44bb-bd15-38f03fcbb017 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:09.188731 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 148/297] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:00:09.194118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b4d5e-ff85-475e-bff0-f9689d0958a8 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:09.194507 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b4d5e-ff85-475e-bff0-f9689d0958a8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:09.194791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b4d5e-ff85-475e-bff0-f9689d0958a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:09.211944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f23b4d5e-ff85-475e-bff0-f9689d0958a8 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:09.213814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f23b4d5e-ff85-475e-bff0-f9689d0958a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:09.249012 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:09.281950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84704de0-f05e-403a-aaf4-9454d8ad6e85 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:09.289719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 149/298] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 15:00:09.292705 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c817a84-4384-4202-a7f7-f629a54a3308 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:09.293001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c817a84-4384-4202-a7f7-f629a54a3308 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:09.293286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c817a84-4384-4202-a7f7-f629a54a3308 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:09.294708 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 150/299] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:09.299942 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc209d-d37c-4cd4-9605-e6ca2c15e790 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:09.300236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc209d-d37c-4cd4-9605-e6ca2c15e790 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:09.300860 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc209d-d37c-4cd4-9605-e6ca2c15e790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:09.320239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-adfc209d-d37c-4cd4-9605-e6ca2c15e790 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:09.321464 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-adfc209d-d37c-4cd4-9605-e6ca2c15e790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:09.322566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279121d-a3b7-4f85-a87b-b8520a399e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:09.323643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279121d-a3b7-4f85-a87b-b8520a399e01 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:09.324119 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279121d-a3b7-4f85-a87b-b8520a399e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:09.324589 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279121d-a3b7-4f85-a87b-b8520a399e01 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:09.334953 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279121d-a3b7-4f85-a87b-b8520a399e01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-c279121d-a3b7-4f85-a87b-b8520a399e01 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:09.342725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:09.343687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:09.344488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:09.344488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:09.344488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:09.344488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:09.344488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:09.344488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:09.344488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:09.344488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:09.344488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:09.344488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:09.344488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 15:00:09.344488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:09.344488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 150/300] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:09.345179 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279121d-a3b7-4f85-a87b-b8520a399e01 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:09.397562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 151/301] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 15:00:09.398551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc209d-d37c-4cd4-9605-e6ca2c15e790 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:09.421604 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:09.469200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279121d-a3b7-4f85-a87b-b8520a399e01 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:09.500924 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:09.507892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc801a-2a7b-40e1-b5d1-b8960ca5aff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:09.508738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc801a-2a7b-40e1-b5d1-b8960ca5aff2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:09.509018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc801a-2a7b-40e1-b5d1-b8960ca5aff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:09.509297 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc801a-2a7b-40e1-b5d1-b8960ca5aff2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:09.510479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc801a-2a7b-40e1-b5d1-b8960ca5aff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:09.511972 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc801a-2a7b-40e1-b5d1-b8960ca5aff2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:09.518358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc209d-d37c-4cd4-9605-e6ca2c15e790 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:09.518483 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 151/302] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:09.518483 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc801a-2a7b-40e1-b5d1-b8960ca5aff2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:09.551440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d1fb7-3487-4a19-860f-529359157851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:09.552035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d1fb7-3487-4a19-860f-529359157851 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:09.552315 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d1fb7-3487-4a19-860f-529359157851 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:09.552594 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d1fb7-3487-4a19-860f-529359157851 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:09.554122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d1fb7-3487-4a19-860f-529359157851 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:09.554425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d1fb7-3487-4a19-860f-529359157851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:09.555849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d1fb7-3487-4a19-860f-529359157851 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:09.569275 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 152/303] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:09.569275 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d1fb7-3487-4a19-860f-529359157851 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:09.628613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc801a-2a7b-40e1-b5d1-b8960ca5aff2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:09.652511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:09.664646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b6bf5-24b3-4196-993c-ba912887afae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:09.665448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b6bf5-24b3-4196-993c-ba912887afae None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:09.665733 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b6bf5-24b3-4196-993c-ba912887afae None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:09.666011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b6bf5-24b3-4196-993c-ba912887afae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:09.676304 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b6bf5-24b3-4196-993c-ba912887afae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:09.678169 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d1fb7-3487-4a19-860f-529359157851 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-b18b6bf5-24b3-4196-993c-ba912887afae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:09.684286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:09.685195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:09.686312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:09.686312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:09.686312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:09.686312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:09.686312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:09.686312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:09.686312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:09.686312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:09.686312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:09.686312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:09.686312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:09.686312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:09.686312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 152/304] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:09.686312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b6bf5-24b3-4196-993c-ba912887afae None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:09.719876 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce2c93-ac64-42cb-8a18-0fd011b9368d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:09.720554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce2c93-ac64-42cb-8a18-0fd011b9368d None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:09.720847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce2c93-ac64-42cb-8a18-0fd011b9368d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:09.721111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce2c93-ac64-42cb-8a18-0fd011b9368d None admin] PATH_INFO: `/v3/groups` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:09.721807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce2c93-ac64-42cb-8a18-0fd011b9368d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:09.723293 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce2c93-ac64-42cb-8a18-0fd011b9368d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:09.752011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:09.777228 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 153/305] 10.222.0.67 () {60 vars in 1134 bytes} [Fri Jun 24 15:00:09 2022] POST /identity/v3/groups => generated 229 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:00:09.778305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce2c93-ac64-42cb-8a18-0fd011b9368d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:09.817582 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b6bf5-24b3-4196-993c-ba912887afae None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:09.873623 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaaa217-6f88-4630-ad2a-e6c62123feef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:09.874955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaaa217-6f88-4630-ad2a-e6c62123feef None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:09.875454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaaa217-6f88-4630-ad2a-e6c62123feef None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:09.875954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaaa217-6f88-4630-ad2a-e6c62123feef None admin] PATH_INFO: `/v3/users/c805fded16604af887de0e1e839b910a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:09.898848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaaa217-6f88-4630-ad2a-e6c62123feef None admin] RBAC: Authorizing `identity:get_user(user_id=c805fded16604af887de0e1e839b910a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:09.901747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaaa217-6f88-4630-ad2a-e6c62123feef None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:09.904852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 153/306] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:00:09 2022] GET /identity/v3/users/c805fded16604af887de0e1e839b910a => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:09.911827 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaaa217-6f88-4630-ad2a-e6c62123feef None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:09.917618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce2c93-ac64-42cb-8a18-0fd011b9368d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:09.966334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bd116-d118-4d32-b014-1c91e89053f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:09.967022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bd116-d118-4d32-b014-1c91e89053f5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:09.967960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bd116-d118-4d32-b014-1c91e89053f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:09.967960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bd116-d118-4d32-b014-1c91e89053f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:09.968646 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117bd116-d118-4d32-b014-1c91e89053f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:09.968646 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117bd116-d118-4d32-b014-1c91e89053f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:09.970056 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117bd116-d118-4d32-b014-1c91e89053f5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:09.972227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:09.983261 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 154/307] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:09.990991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bd116-d118-4d32-b014-1c91e89053f5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:10.045187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaaa217-6f88-4630-ad2a-e6c62123feef None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:10.063927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:10.083532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae430b33-e312-4d1a-8a2d-257d68e970df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:10.084531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae430b33-e312-4d1a-8a2d-257d68e970df None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:10.084982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae430b33-e312-4d1a-8a2d-257d68e970df None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:10.085607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae430b33-e312-4d1a-8a2d-257d68e970df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:10.086589 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae430b33-e312-4d1a-8a2d-257d68e970df None admin] RBAC: Adding query filter params (role.id=ae37f175be764624b8ef6f3c0104dbf9, user.id=c805fded16604af887de0e1e839b910a, scope.system=all) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:10.087042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae430b33-e312-4d1a-8a2d-257d68e970df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:10.088677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae430b33-e312-4d1a-8a2d-257d68e970df None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:10.095523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 154/308] 10.222.0.67 () {56 vars in 1308 bytes} [Fri Jun 24 15:00:09 2022] GET /identity/v3/role_assignments?user.id=c805fded16604af887de0e1e839b910a&scope.system=all&role.id=ae37f175be764624b8ef6f3c0104dbf9 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:10.096786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bdd3e8-124f-4022-ac66-59b1c7b5fff4 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:10.097075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bdd3e8-124f-4022-ac66-59b1c7b5fff4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:10.097350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bdd3e8-124f-4022-ac66-59b1c7b5fff4 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:10.098781 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 155/309] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:10.104397 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3792a-3163-4da5-a483-6695f0e17c77 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:10.104686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3792a-3163-4da5-a483-6695f0e17c77 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:10.104945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3792a-3163-4da5-a483-6695f0e17c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:10.122917 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3fe3792a-3163-4da5-a483-6695f0e17c77 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:10.124310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3fe3792a-3163-4da5-a483-6695f0e17c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:10.128067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bd116-d118-4d32-b014-1c91e89053f5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:10.174177 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a8b82-6edd-450e-a21e-1dafe52e1075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:10.175873 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a8b82-6edd-450e-a21e-1dafe52e1075 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:10.176160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a8b82-6edd-450e-a21e-1dafe52e1075 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:10.176433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a8b82-6edd-450e-a21e-1dafe52e1075 None admin] PATH_INFO: `/v3/users/6edee7cc751f41d6a9dff0779f260069` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:10.179243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a8b82-6edd-450e-a21e-1dafe52e1075 None admin] RBAC: Authorizing `identity:get_user(user_id=6edee7cc751f41d6a9dff0779f260069)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:10.180748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a8b82-6edd-450e-a21e-1dafe52e1075 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:10.183150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 155/310] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:00:09 2022] GET /identity/v3/users/6edee7cc751f41d6a9dff0779f260069 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:10.188322 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a8b82-6edd-450e-a21e-1dafe52e1075 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:10.211564 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 156/311] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 15:00:10.231444 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3792a-3163-4da5-a483-6695f0e17c77 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:10.272433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:10.322239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a8b82-6edd-450e-a21e-1dafe52e1075 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:10.350345 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3792a-3163-4da5-a483-6695f0e17c77 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:10.362772 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:10.368214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb435d3e-7c53-473c-9c28-59b034d92868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:10.369451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb435d3e-7c53-473c-9c28-59b034d92868 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:10.369880 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb435d3e-7c53-473c-9c28-59b034d92868 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:10.370586 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb435d3e-7c53-473c-9c28-59b034d92868 None admin] PATH_INFO: `/v3/system/users/6edee7cc751f41d6a9dff0779f260069/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:10.372321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb435d3e-7c53-473c-9c28-59b034d92868 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6edee7cc751f41d6a9dff0779f260069, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:10.374414 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb435d3e-7c53-473c-9c28-59b034d92868 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:10.389369 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36761c09-db8e-46eb-82df-62ab30c4c11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:10.390415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36761c09-db8e-46eb-82df-62ab30c4c11f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:10.390722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36761c09-db8e-46eb-82df-62ab30c4c11f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:10.390982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36761c09-db8e-46eb-82df-62ab30c4c11f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:10.402952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36761c09-db8e-46eb-82df-62ab30c4c11f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-36761c09-db8e-46eb-82df-62ab30c4c11f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:10.410895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:10.411844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:10.412662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:10.412662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:10.412662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:10.412662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:10.412662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:10.412662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:10.412662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:10.412662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:10.412662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:10.412662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:10.412662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 15:00:10.412662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:10.412662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 157/312] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:10.419284 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36761c09-db8e-46eb-82df-62ab30c4c11f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:10.433676 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 156/313] 10.222.0.67 () {58 vars in 1334 bytes} [Fri Jun 24 15:00:10 2022] PUT /identity/v3/system/users/6edee7cc751f41d6a9dff0779f260069/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:00:10.535515 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36761c09-db8e-46eb-82df-62ab30c4c11f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:10.574688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50115564-5277-4015-818a-37d8eee49016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:10.575705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50115564-5277-4015-818a-37d8eee49016 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:10.576164 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50115564-5277-4015-818a-37d8eee49016 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:10.576589 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50115564-5277-4015-818a-37d8eee49016 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:10.577524 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50115564-5277-4015-818a-37d8eee49016 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:10.577988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50115564-5277-4015-818a-37d8eee49016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:10.579582 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50115564-5277-4015-818a-37d8eee49016 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:10.590071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 158/314] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:10.595809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb435d3e-7c53-473c-9c28-59b034d92868 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:10.703520 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb435d3e-7c53-473c-9c28-59b034d92868 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:10.730962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:10.739577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c0816-a990-4f39-bc94-3e3a6f38f1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:10.742709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c0816-a990-4f39-bc94-3e3a6f38f1db None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:10.742709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c0816-a990-4f39-bc94-3e3a6f38f1db None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:10.742709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c0816-a990-4f39-bc94-3e3a6f38f1db None admin] PATH_INFO: `/v3/users/53a386e88ed14336a5c8c46aa95bdd74` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:10.743002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87c0816-a990-4f39-bc94-3e3a6f38f1db None admin] RBAC: Authorizing `identity:get_user(user_id=53a386e88ed14336a5c8c46aa95bdd74)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:10.744350 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87c0816-a990-4f39-bc94-3e3a6f38f1db None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:10.746018 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 157/315] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:00:10 2022] GET /identity/v3/users/53a386e88ed14336a5c8c46aa95bdd74 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:10.754063 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50115564-5277-4015-818a-37d8eee49016 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:10.788760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:10.863408 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50115564-5277-4015-818a-37d8eee49016 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:10.900618 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa2631-6db1-49d7-96dd-3d993d3ae3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:10.901280 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa2631-6db1-49d7-96dd-3d993d3ae3ff None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:10.901566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa2631-6db1-49d7-96dd-3d993d3ae3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:10.901845 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa2631-6db1-49d7-96dd-3d993d3ae3ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:10.973100 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa2631-6db1-49d7-96dd-3d993d3ae3ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:10.974596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa2631-6db1-49d7-96dd-3d993d3ae3ff None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:11.005384 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:11.078810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 159/316] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:11.084896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c0816-a990-4f39-bc94-3e3a6f38f1db None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:11.191451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c0816-a990-4f39-bc94-3e3a6f38f1db None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:11.228168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc98fa9-6cb4-4d69-a51e-543a6a73000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:11.229376 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc98fa9-6cb4-4d69-a51e-543a6a73000c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:11.229878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc98fa9-6cb4-4d69-a51e-543a6a73000c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:11.230358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc98fa9-6cb4-4d69-a51e-543a6a73000c None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:11.231779 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc98fa9-6cb4-4d69-a51e-543a6a73000c None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:11.233424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc98fa9-6cb4-4d69-a51e-543a6a73000c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:11.235728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 158/317] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:11 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:11.241275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa2631-6db1-49d7-96dd-3d993d3ae3ff None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:11.347589 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa2631-6db1-49d7-96dd-3d993d3ae3ff None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:11.378062 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:11.383084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811e44b-b510-4ee2-afd1-7140c4d0fc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:11.383778 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811e44b-b510-4ee2-afd1-7140c4d0fc45 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:11.384066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811e44b-b510-4ee2-afd1-7140c4d0fc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:11.384346 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811e44b-b510-4ee2-afd1-7140c4d0fc45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:11.385183 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811e44b-b510-4ee2-afd1-7140c4d0fc45 None admin] RBAC: Adding query filter params (role.id=ae37f175be764624b8ef6f3c0104dbf9, scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487, user.id=53a386e88ed14336a5c8c46aa95bdd74) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:11.385497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811e44b-b510-4ee2-afd1-7140c4d0fc45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:11.386994 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811e44b-b510-4ee2-afd1-7140c4d0fc45 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:11.393219 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 160/318] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:11 2022] GET /identity/v3/role_assignments?user.id=53a386e88ed14336a5c8c46aa95bdd74&scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487&role.id=ae37f175be764624b8ef6f3c0104dbf9 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:11.984044 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8dc2a-ca69-4a04-894e-9db246a5a8f2 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:11.984420 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8dc2a-ca69-4a04-894e-9db246a5a8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:11.984615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8dc2a-ca69-4a04-894e-9db246a5a8f2 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:11.986114 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 159/319] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:11.991927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72531ca3-91db-481d-9db8-294c7c88e843 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:11.992218 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72531ca3-91db-481d-9db8-294c7c88e843 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:11.992501 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72531ca3-91db-481d-9db8-294c7c88e843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:12.009245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-72531ca3-91db-481d-9db8-294c7c88e843 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:12.010431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-72531ca3-91db-481d-9db8-294c7c88e843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:12.080292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 161/320] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:12.104106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8dc2a-ca69-4a04-894e-9db246a5a8f2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:12.170018 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:12.196308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ddc6e-4eda-41f2-8cf7-0cce01e83792 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:12.196626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ddc6e-4eda-41f2-8cf7-0cce01e83792 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:12.196873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ddc6e-4eda-41f2-8cf7-0cce01e83792 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:12.198259 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 162/321] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:12.203111 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec767e4d-6d7c-4049-aac6-703019a58a0f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:12.203445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec767e4d-6d7c-4049-aac6-703019a58a0f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:12.203733 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec767e4d-6d7c-4049-aac6-703019a58a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:12.210660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8dc2a-ca69-4a04-894e-9db246a5a8f2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:12.220567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ec767e4d-6d7c-4049-aac6-703019a58a0f None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:12.221745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ec767e4d-6d7c-4049-aac6-703019a58a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:12.246096 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428c761-ec75-41b4-8698-9220193dbc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:12.246820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428c761-ec75-41b4-8698-9220193dbc63 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:12.247102 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428c761-ec75-41b4-8698-9220193dbc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:12.247405 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428c761-ec75-41b4-8698-9220193dbc63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:12.248554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428c761-ec75-41b4-8698-9220193dbc63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:12.249990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428c761-ec75-41b4-8698-9220193dbc63 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:12.256013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 160/322] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:12.257004 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428c761-ec75-41b4-8698-9220193dbc63 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:12.296165 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 163/323] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 15:00:12.318679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec767e4d-6d7c-4049-aac6-703019a58a0f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:12.360011 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428c761-ec75-41b4-8698-9220193dbc63 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:12.374044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:12.391443 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:12.397143 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc8a9a-c192-4b7c-b1db-fa6a7b58601d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:12.397817 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc8a9a-c192-4b7c-b1db-fa6a7b58601d None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:12.398101 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc8a9a-c192-4b7c-b1db-fa6a7b58601d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:12.398377 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc8a9a-c192-4b7c-b1db-fa6a7b58601d None admin] PATH_INFO: `/v3/groups` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:12.399068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc8a9a-c192-4b7c-b1db-fa6a7b58601d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:12.400541 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc8a9a-c192-4b7c-b1db-fa6a7b58601d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:12.418325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec767e4d-6d7c-4049-aac6-703019a58a0f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:12.450785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740e683-c8c8-4a00-b3e5-5393c6316dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:12.451628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740e683-c8c8-4a00-b3e5-5393c6316dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:12.451910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740e683-c8c8-4a00-b3e5-5393c6316dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:12.452182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740e683-c8c8-4a00-b3e5-5393c6316dd9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:12.462967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740e683-c8c8-4a00-b3e5-5393c6316dd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-b740e683-c8c8-4a00-b3e5-5393c6316dd9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:12.470645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:12.471805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:12.472710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:12.472710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:12.472710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:12.472710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:12.472710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:12.472710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:12.472710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:12.472710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:12.472710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:12.472710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:12.472710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 15:00:12.472710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:12.472710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 164/324] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:12.478473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740e683-c8c8-4a00-b3e5-5393c6316dd9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:12.495252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 161/325] 10.222.0.67 () {60 vars in 1134 bytes} [Fri Jun 24 15:00:12 2022] POST /identity/v3/groups => generated 226 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:00:12.598618 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740e683-c8c8-4a00-b3e5-5393c6316dd9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:12.616560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f8747-91b4-405e-819c-2daa3da71180 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:12.616915 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f8747-91b4-405e-819c-2daa3da71180 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:12.617187 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f8747-91b4-405e-819c-2daa3da71180 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:12.618483 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:12.618979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 162/326] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:00:12.624147 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1637e-60a3-46a6-8c4e-f06275c96d2b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:12.626821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1637e-60a3-46a6-8c4e-f06275c96d2b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:12.626821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1637e-60a3-46a6-8c4e-f06275c96d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:12.638647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4caf4-59e0-42bc-9afe-0a299fceda57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:12.639377 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4caf4-59e0-42bc-9afe-0a299fceda57 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:12.640476 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4caf4-59e0-42bc-9afe-0a299fceda57 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:12.640476 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4caf4-59e0-42bc-9afe-0a299fceda57 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:12.640651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4caf4-59e0-42bc-9afe-0a299fceda57 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:12.640965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4caf4-59e0-42bc-9afe-0a299fceda57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:12.642351 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4caf4-59e0-42bc-9afe-0a299fceda57 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:12.642937 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-84f1637e-60a3-46a6-8c4e-f06275c96d2b None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:12.644112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-84f1637e-60a3-46a6-8c4e-f06275c96d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:12.652967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 165/327] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:12.659096 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4caf4-59e0-42bc-9afe-0a299fceda57 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:12.735822 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 163/328] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 15:00:12.761707 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1637e-60a3-46a6-8c4e-f06275c96d2b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:12.773379 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4caf4-59e0-42bc-9afe-0a299fceda57 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:12.812354 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590c0da-3081-4309-8e5d-e8d70426a754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:12.814084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590c0da-3081-4309-8e5d-e8d70426a754 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:12.814377 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590c0da-3081-4309-8e5d-e8d70426a754 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:12.814661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590c0da-3081-4309-8e5d-e8d70426a754 None admin] PATH_INFO: `/v3/users/c805fded16604af887de0e1e839b910a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:12.815932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590c0da-3081-4309-8e5d-e8d70426a754 None admin] RBAC: Authorizing `identity:get_user(user_id=c805fded16604af887de0e1e839b910a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:12.817401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590c0da-3081-4309-8e5d-e8d70426a754 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:12.824546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 166/329] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:00:12 2022] GET /identity/v3/users/c805fded16604af887de0e1e839b910a => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:12.827786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590c0da-3081-4309-8e5d-e8d70426a754 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:12.888366 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:12.902073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1637e-60a3-46a6-8c4e-f06275c96d2b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:12.932929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:12.948319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf8f0e-db85-4c0a-89ba-18a253230638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:12.949358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf8f0e-db85-4c0a-89ba-18a253230638 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:12.949785 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf8f0e-db85-4c0a-89ba-18a253230638 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:12.950205 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf8f0e-db85-4c0a-89ba-18a253230638 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:12.961644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf8f0e-db85-4c0a-89ba-18a253230638 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:12.964933 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590c0da-3081-4309-8e5d-e8d70426a754 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-8fdf8f0e-db85-4c0a-89ba-18a253230638 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:12.970431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:12.971546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:12.972492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:12.972492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:12.972492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:12.972492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:12.972492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:12.972492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:12.972492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:12.972492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:12.972492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:12.972492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:12.972492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:12.972492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:12.975494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 164/330] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:12.984244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf8f0e-db85-4c0a-89ba-18a253230638 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:13.012720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d048a0-c1b4-49fb-8409-1935206b6f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:13.013644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d048a0-c1b4-49fb-8409-1935206b6f2c None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:13.013922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d048a0-c1b4-49fb-8409-1935206b6f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:13.015523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d048a0-c1b4-49fb-8409-1935206b6f2c None admin] PATH_INFO: `/v3/system/users/c805fded16604af887de0e1e839b910a/roles/ae37f175be764624b8ef6f3c0104dbf9` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:13.018049 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d048a0-c1b4-49fb-8409-1935206b6f2c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c805fded16604af887de0e1e839b910a, role_id=ae37f175be764624b8ef6f3c0104dbf9)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:13.019757 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d048a0-c1b4-49fb-8409-1935206b6f2c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:13.062608 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 167/331] 10.222.0.67 () {58 vars in 1334 bytes} [Fri Jun 24 15:00:12 2022] PUT /identity/v3/system/users/c805fded16604af887de0e1e839b910a/roles/ae37f175be764624b8ef6f3c0104dbf9 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:00:13.120420 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf8f0e-db85-4c0a-89ba-18a253230638 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:13.158749 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7d214-92ab-4e2c-b060-3a437a29dc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:13.159395 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7d214-92ab-4e2c-b060-3a437a29dc4c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:13.159682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7d214-92ab-4e2c-b060-3a437a29dc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:13.159959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7d214-92ab-4e2c-b060-3a437a29dc4c None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:13.160812 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7d214-92ab-4e2c-b060-3a437a29dc4c None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:13.161128 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7d214-92ab-4e2c-b060-3a437a29dc4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:13.163193 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7d214-92ab-4e2c-b060-3a437a29dc4c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:13.173452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 165/332] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:13.179343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d048a0-c1b4-49fb-8409-1935206b6f2c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:13.295506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d048a0-c1b4-49fb-8409-1935206b6f2c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:13.307442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:13.332883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555100f1-b82f-4ed9-b5d1-60cc13ac3820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:13.334529 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555100f1-b82f-4ed9-b5d1-60cc13ac3820 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:13.334739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555100f1-b82f-4ed9-b5d1-60cc13ac3820 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:13.335025 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555100f1-b82f-4ed9-b5d1-60cc13ac3820 None admin] PATH_INFO: `/v3/users/6edee7cc751f41d6a9dff0779f260069` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:13.336485 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555100f1-b82f-4ed9-b5d1-60cc13ac3820 None admin] RBAC: Authorizing `identity:get_user(user_id=6edee7cc751f41d6a9dff0779f260069)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:13.337936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555100f1-b82f-4ed9-b5d1-60cc13ac3820 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:13.339933 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 168/333] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:00:13 2022] GET /identity/v3/users/6edee7cc751f41d6a9dff0779f260069 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:13.345534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7d214-92ab-4e2c-b060-3a437a29dc4c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:13.397645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:13.454185 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7d214-92ab-4e2c-b060-3a437a29dc4c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:13.489462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349d06b-52da-4401-b205-152983a712a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:13.490302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349d06b-52da-4401-b205-152983a712a6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:13.490546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349d06b-52da-4401-b205-152983a712a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:13.490712 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349d06b-52da-4401-b205-152983a712a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:13.491589 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349d06b-52da-4401-b205-152983a712a6 None admin] RBAC: Adding query filter params (role.id=b6dd71d4043d43e9823ec25176863347, user.id=6edee7cc751f41d6a9dff0779f260069, scope.system=all) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:13.491904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349d06b-52da-4401-b205-152983a712a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:13.493351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349d06b-52da-4401-b205-152983a712a6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:13.506544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 166/334] 10.222.0.67 () {56 vars in 1308 bytes} [Fri Jun 24 15:00:13 2022] GET /identity/v3/role_assignments?user.id=6edee7cc751f41d6a9dff0779f260069&scope.system=all&role.id=b6dd71d4043d43e9823ec25176863347 => generated 510 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:15.109818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57778c-b773-42cb-a0d6-c6d65c533b90 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:15.110686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57778c-b773-42cb-a0d6-c6d65c533b90 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:15.111124 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57778c-b773-42cb-a0d6-c6d65c533b90 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:15.112963 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 169/335] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:15.118566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce602cee-976b-4d60-a14a-62621df2356a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:15.119449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce602cee-976b-4d60-a14a-62621df2356a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:15.119910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce602cee-976b-4d60-a14a-62621df2356a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:15.144254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ce602cee-976b-4d60-a14a-62621df2356a None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:15.145754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ce602cee-976b-4d60-a14a-62621df2356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:15.228150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 167/336] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:15.263225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57778c-b773-42cb-a0d6-c6d65c533b90 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:15.388211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57778c-b773-42cb-a0d6-c6d65c533b90 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:15.428656 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800a945-c864-49cc-a73d-9647863e7cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:15.429771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800a945-c864-49cc-a73d-9647863e7cff None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:15.430259 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800a945-c864-49cc-a73d-9647863e7cff None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:15.430722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800a945-c864-49cc-a73d-9647863e7cff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:15.441497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800a945-c864-49cc-a73d-9647863e7cff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f800a945-c864-49cc-a73d-9647863e7cff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:15.450544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:15.451495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:15.452296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:15.452296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:15.452296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:15.452296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:15.452296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:15.452296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:15.452296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:15.452296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:15.452296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:15.452296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:15.452296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:15.452296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:15.454316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 170/337] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:00:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:15.461366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce602cee-976b-4d60-a14a-62621df2356a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:15.500228 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:15.568759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce602cee-976b-4d60-a14a-62621df2356a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:15.590273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068947a-a87f-480d-aaae-1ef8e800667e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:15.590563 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068947a-a87f-480d-aaae-1ef8e800667e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:15.591041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068947a-a87f-480d-aaae-1ef8e800667e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:15.592832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 171/338] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:15.599411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07131962-7c65-4c52-aaf1-d980b8a51058 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:15.600746 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07131962-7c65-4c52-aaf1-d980b8a51058 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:15.601257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07131962-7c65-4c52-aaf1-d980b8a51058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:15.606895 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b9c2c-bd5f-4f40-ad37-68db0cfab323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:15.608296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b9c2c-bd5f-4f40-ad37-68db0cfab323 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:15.608296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b9c2c-bd5f-4f40-ad37-68db0cfab323 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:15.608296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b9c2c-bd5f-4f40-ad37-68db0cfab323 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:15.608876 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b9c2c-bd5f-4f40-ad37-68db0cfab323 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:15.609180 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b9c2c-bd5f-4f40-ad37-68db0cfab323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:15.610539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b9c2c-bd5f-4f40-ad37-68db0cfab323 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:15.619512 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 168/339] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:15.625902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b9c2c-bd5f-4f40-ad37-68db0cfab323 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:15.645861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-07131962-7c65-4c52-aaf1-d980b8a51058 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:15.647695 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-07131962-7c65-4c52-aaf1-d980b8a51058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:15.699667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:15.773575 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 172/340] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 15:00:15.775293 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475e0c7-b337-482e-af05-caf33393eb4b None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:15.775723 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475e0c7-b337-482e-af05-caf33393eb4b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:15.776138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475e0c7-b337-482e-af05-caf33393eb4b None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:15.777841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 173/341] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:15.783989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a888d-a2d3-4fb0-9416-46eb0d04b835 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:15.784594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a888d-a2d3-4fb0-9416-46eb0d04b835 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:15.785015 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a888d-a2d3-4fb0-9416-46eb0d04b835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:15.786216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:15.801319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b9c2c-bd5f-4f40-ad37-68db0cfab323 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:15.804669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fc5a888d-a2d3-4fb0-9416-46eb0d04b835 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:15.806053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fc5a888d-a2d3-4fb0-9416-46eb0d04b835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:15.871292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623b59a-1685-4691-9946-9717a5dc13eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:15.872711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623b59a-1685-4691-9946-9717a5dc13eb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:15.874388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623b59a-1685-4691-9946-9717a5dc13eb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:15.874844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623b59a-1685-4691-9946-9717a5dc13eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:15.915704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 174/342] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 15:00:15.917862 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a888d-a2d3-4fb0-9416-46eb0d04b835 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:16.009124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9623b59a-1685-4691-9946-9717a5dc13eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:16.010822 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9623b59a-1685-4691-9946-9717a5dc13eb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:16.069005 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:16.073412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a888d-a2d3-4fb0-9416-46eb0d04b835 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:16.108983 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900ddec-f6ba-4bbc-b45c-b690075da314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:16.110002 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900ddec-f6ba-4bbc-b45c-b690075da314 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:16.110869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900ddec-f6ba-4bbc-b45c-b690075da314 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:16.111327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900ddec-f6ba-4bbc-b45c-b690075da314 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:16.122552 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900ddec-f6ba-4bbc-b45c-b690075da314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:16.123153 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:16.131348 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 169/343] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8900ddec-f6ba-4bbc-b45c-b690075da314 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:16.132073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:16.133142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:16.134172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:16.134172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:16.134172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:16.134172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:16.134172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:16.134172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:16.134172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:16.134172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:16.134172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:16.134172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:16.134172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:16.134172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:16.134819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623b59a-1685-4691-9946-9717a5dc13eb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:16.140964 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 175/344] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:16.142519 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62151e13-86ef-4f03-923b-5f86da03bb93 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:16.142953 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62151e13-86ef-4f03-923b-5f86da03bb93 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:16.147529 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62151e13-86ef-4f03-923b-5f86da03bb93 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:16.147529 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 176/345] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:16.147529 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62151e13-86ef-4f03-923b-5f86da03bb93 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:16.246161 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623b59a-1685-4691-9946-9717a5dc13eb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:16.254551 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62151e13-86ef-4f03-923b-5f86da03bb93 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:16.283813 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af431c6-b956-4bd5-a387-1c8887ffb9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:16.284897 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af431c6-b956-4bd5-a387-1c8887ffb9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:16.285325 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af431c6-b956-4bd5-a387-1c8887ffb9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:16.285839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af431c6-b956-4bd5-a387-1c8887ffb9f1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:16.292430 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4d55e-257d-4acb-9d6a-6f764e4b6fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:16.293706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4d55e-257d-4acb-9d6a-6f764e4b6fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:16.294146 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4d55e-257d-4acb-9d6a-6f764e4b6fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:16.294563 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4d55e-257d-4acb-9d6a-6f764e4b6fa0 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:16.296001 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4d55e-257d-4acb-9d6a-6f764e4b6fa0 None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:16.297596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4d55e-257d-4acb-9d6a-6f764e4b6fa0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:16.298452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af431c6-b956-4bd5-a387-1c8887ffb9f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:16.299665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 177/346] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:16 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:16.301153 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32593c12-10f6-4f23-9c0d-985e251fa222 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:16.301580 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32593c12-10f6-4f23-9c0d-985e251fa222 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:16.301996 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32593c12-10f6-4f23-9c0d-985e251fa222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-5af431c6-b956-4bd5-a387-1c8887ffb9f1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:16.308179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:16.309321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:16.310146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:16.310146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:16.310146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:16.310146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:16.310146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:16.310146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:16.310146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:16.310146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:16.310146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:16.310146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:16.310146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 15:00:16.310146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:16.310929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 170/347] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:16.312037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af431c6-b956-4bd5-a387-1c8887ffb9f1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:16.324400 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-32593c12-10f6-4f23-9c0d-985e251fa222 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:16.325906 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-32593c12-10f6-4f23-9c0d-985e251fa222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:16.347953 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:16.376913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:16.407909 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 178/348] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:16.412393 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32593c12-10f6-4f23-9c0d-985e251fa222 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:16.457087 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af431c6-b956-4bd5-a387-1c8887ffb9f1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:16.523038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd85047-9b9b-4691-9ee9-4cbd1ba110ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:16.523674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd85047-9b9b-4691-9ee9-4cbd1ba110ad None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:16.523956 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd85047-9b9b-4691-9ee9-4cbd1ba110ad None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:16.524231 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd85047-9b9b-4691-9ee9-4cbd1ba110ad None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:16.524979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd85047-9b9b-4691-9ee9-4cbd1ba110ad None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:16.525289 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd85047-9b9b-4691-9ee9-4cbd1ba110ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:16.526662 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd85047-9b9b-4691-9ee9-4cbd1ba110ad None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:16.541440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 171/349] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:16.546139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd85047-9b9b-4691-9ee9-4cbd1ba110ad None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:16.565197 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32593c12-10f6-4f23-9c0d-985e251fa222 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:16.606991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756611a-184e-45e1-ab0c-82510f67c24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:16.607925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756611a-184e-45e1-ab0c-82510f67c24c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:16.608210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756611a-184e-45e1-ab0c-82510f67c24c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:16.608463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756611a-184e-45e1-ab0c-82510f67c24c None admin] PATH_INFO: `/v3/groups/d5308803b7ea4a8499209c30aa8d030d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:16.617040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756611a-184e-45e1-ab0c-82510f67c24c None admin] RBAC: Authorizing `identity:get_group(group_id=d5308803b7ea4a8499209c30aa8d030d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:16.618687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756611a-184e-45e1-ab0c-82510f67c24c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:16.620831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 179/350] 10.222.0.67 () {56 vars in 1180 bytes} [Fri Jun 24 15:00:16 2022] GET /identity/v3/groups/d5308803b7ea4a8499209c30aa8d030d => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:16.621599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756611a-184e-45e1-ab0c-82510f67c24c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:16.647691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:16.674991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd85047-9b9b-4691-9ee9-4cbd1ba110ad None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:16.697619 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:16.719488 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878d89b-7b43-41fe-ab89-163478271eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:16.720893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878d89b-7b43-41fe-ab89-163478271eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:16.721344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878d89b-7b43-41fe-ab89-163478271eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:16.721776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878d89b-7b43-41fe-ab89-163478271eb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:16.723268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878d89b-7b43-41fe-ab89-163478271eb3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:16.723723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878d89b-7b43-41fe-ab89-163478271eb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:16.725974 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878d89b-7b43-41fe-ab89-163478271eb3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:16.736748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 172/351] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:16.737776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878d89b-7b43-41fe-ab89-163478271eb3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:16.754443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756611a-184e-45e1-ab0c-82510f67c24c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:16.800508 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51701e-4e99-47a9-9562-81f8fd933b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:16.801950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51701e-4e99-47a9-9562-81f8fd933b11 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:16.802423 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51701e-4e99-47a9-9562-81f8fd933b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:16.802957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51701e-4e99-47a9-9562-81f8fd933b11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:16.816360 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51701e-4e99-47a9-9562-81f8fd933b11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ad51701e-4e99-47a9-9562-81f8fd933b11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:16.826841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:16.827808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:16.828689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:16.828689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:16.828689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:16.828689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:16.828689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:16.828689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:16.828689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:16.828689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:16.828689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:16.828689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:16.828689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:16.828689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:16.831503 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 180/352] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:16.832828 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51701e-4e99-47a9-9562-81f8fd933b11 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:16.865563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878d89b-7b43-41fe-ab89-163478271eb3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:16.912452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72dc62-abee-4d47-9f94-53335d96efea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:16.914140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72dc62-abee-4d47-9f94-53335d96efea None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:16.914598 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72dc62-abee-4d47-9f94-53335d96efea None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:16.915017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72dc62-abee-4d47-9f94-53335d96efea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:16.934710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:16.957121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51701e-4e99-47a9-9562-81f8fd933b11 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:16.977827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:16.997869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dccdd0-b93a-435f-8c2d-0a54b60b6991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:16.998640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dccdd0-b93a-435f-8c2d-0a54b60b6991 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:16.998928 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dccdd0-b93a-435f-8c2d-0a54b60b6991 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:16.999218 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dccdd0-b93a-435f-8c2d-0a54b60b6991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:17.000067 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dccdd0-b93a-435f-8c2d-0a54b60b6991 None admin] RBAC: Adding query filter params (group.id=d5308803b7ea4a8499209c30aa8d030d, role.id=c077ebdf7f214d7585dfc1570eecf882, scope.project.id=4d31d318708e4e9caa66106ce370a235) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:17.000387 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dccdd0-b93a-435f-8c2d-0a54b60b6991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:17.001878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dccdd0-b93a-435f-8c2d-0a54b60b6991 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:17.008573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 181/353] 10.222.0.67 () {56 vars in 1376 bytes} [Fri Jun 24 15:00:16 2022] GET /identity/v3/role_assignments?group.id=d5308803b7ea4a8499209c30aa8d030d&scope.project.id=4d31d318708e4e9caa66106ce370a235&role.id=c077ebdf7f214d7585dfc1570eecf882 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:17.009235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dccdd0-b93a-435f-8c2d-0a54b60b6991 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:17.012744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72dc62-abee-4d47-9f94-53335d96efea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:17.016373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72dc62-abee-4d47-9f94-53335d96efea None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:17.139577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 173/354] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:17.140596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72dc62-abee-4d47-9f94-53335d96efea None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:17.171011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dccdd0-b93a-435f-8c2d-0a54b60b6991 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:17.211776 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783108de-820f-4c27-93b1-f688273366da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:17.212839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783108de-820f-4c27-93b1-f688273366da None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:17.213485 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783108de-820f-4c27-93b1-f688273366da None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:17.213772 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783108de-820f-4c27-93b1-f688273366da None admin] PATH_INFO: `/v3/users/c805fded16604af887de0e1e839b910a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:17.215014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783108de-820f-4c27-93b1-f688273366da None admin] RBAC: Authorizing `identity:get_user(user_id=c805fded16604af887de0e1e839b910a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:17.216509 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783108de-820f-4c27-93b1-f688273366da None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:17.218267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 182/355] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:00:17 2022] GET /identity/v3/users/c805fded16604af887de0e1e839b910a => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:17.219703 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11e2fd-e53a-4a75-8bab-e45818612eea None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:17.220000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11e2fd-e53a-4a75-8bab-e45818612eea None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:17.220278 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11e2fd-e53a-4a75-8bab-e45818612eea None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:17.221642 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 183/356] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:17.222245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11e2fd-e53a-4a75-8bab-e45818612eea None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:17.239109 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:17.243567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:17.274962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72dc62-abee-4d47-9f94-53335d96efea None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:17.322601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e25c6-4767-4850-923d-76afc789902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:17.323603 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e25c6-4767-4850-923d-76afc789902f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:17.324049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e25c6-4767-4850-923d-76afc789902f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:17.324467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e25c6-4767-4850-923d-76afc789902f None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:17.325370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e25c6-4767-4850-923d-76afc789902f None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:17.325807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e25c6-4767-4850-923d-76afc789902f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:17.327369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e25c6-4767-4850-923d-76afc789902f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:17.337389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 174/357] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:17.338414 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e25c6-4767-4850-923d-76afc789902f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:17.343157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11e2fd-e53a-4a75-8bab-e45818612eea None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:17.381084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d28421-c13d-4f6c-973e-59123a417bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:17.381084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d28421-c13d-4f6c-973e-59123a417bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:17.381427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d28421-c13d-4f6c-973e-59123a417bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:17.381835 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d28421-c13d-4f6c-973e-59123a417bf8 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:17.383389 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d28421-c13d-4f6c-973e-59123a417bf8 None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:17.385803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d28421-c13d-4f6c-973e-59123a417bf8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:17.387812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 184/358] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:17 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:17.388892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db2785-af04-4d90-b5cd-4bf00e6d7cb5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:17.389248 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db2785-af04-4d90-b5cd-4bf00e6d7cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:17.389594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db2785-af04-4d90-b5cd-4bf00e6d7cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:17.410392 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d9db2785-af04-4d90-b5cd-4bf00e6d7cb5 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:17.412517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d9db2785-af04-4d90-b5cd-4bf00e6d7cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:17.476659 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e25c6-4767-4850-923d-76afc789902f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:17.513111 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 185/359] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:17.515227 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db2785-af04-4d90-b5cd-4bf00e6d7cb5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:17.523810 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96937895-939d-4a2a-b508-6f7162ea1b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:17.525091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96937895-939d-4a2a-b508-6f7162ea1b50 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:17.525543 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96937895-939d-4a2a-b508-6f7162ea1b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:17.526007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:17.526250 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96937895-939d-4a2a-b508-6f7162ea1b50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:17.527287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96937895-939d-4a2a-b508-6f7162ea1b50 None admin] RBAC: Adding query filter params (role.id=ae37f175be764624b8ef6f3c0104dbf9, user.id=c805fded16604af887de0e1e839b910a, scope.system=all) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:17.527727 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96937895-939d-4a2a-b508-6f7162ea1b50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:17.529190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96937895-939d-4a2a-b508-6f7162ea1b50 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:17.536500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:17.547672 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 175/360] 10.222.0.67 () {56 vars in 1308 bytes} [Fri Jun 24 15:00:17 2022] GET /identity/v3/role_assignments?user.id=c805fded16604af887de0e1e839b910a&scope.system=all&role.id=ae37f175be764624b8ef6f3c0104dbf9 => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:17.548693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96937895-939d-4a2a-b508-6f7162ea1b50 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:17.649240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db2785-af04-4d90-b5cd-4bf00e6d7cb5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:17.689425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96937895-939d-4a2a-b508-6f7162ea1b50 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:17.693225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7658bc-419d-4f02-a7d4-8275026f696b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:17.693892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7658bc-419d-4f02-a7d4-8275026f696b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:17.694173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7658bc-419d-4f02-a7d4-8275026f696b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:17.694450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7658bc-419d-4f02-a7d4-8275026f696b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:17.734300 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458b343-e80e-4069-b228-2c0167e8fbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:17.735294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458b343-e80e-4069-b228-2c0167e8fbff None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:17.735632 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458b343-e80e-4069-b228-2c0167e8fbff None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:17.735926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458b343-e80e-4069-b228-2c0167e8fbff None admin] PATH_INFO: `/v3/groups/d5308803b7ea4a8499209c30aa8d030d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:17.737097 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458b343-e80e-4069-b228-2c0167e8fbff None admin] RBAC: Authorizing `identity:get_group(group_id=d5308803b7ea4a8499209c30aa8d030d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:17.738566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458b343-e80e-4069-b228-2c0167e8fbff None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:17.740273 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 176/361] 10.222.0.67 () {56 vars in 1180 bytes} [Fri Jun 24 15:00:17 2022] GET /identity/v3/groups/d5308803b7ea4a8499209c30aa8d030d => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:17.741680 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d579e-fd60-4d4e-b330-c546ccc737c8 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:17.741965 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d579e-fd60-4d4e-b330-c546ccc737c8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:17.742239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d579e-fd60-4d4e-b330-c546ccc737c8 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:17.743548 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 177/362] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:17.744271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d579e-fd60-4d4e-b330-c546ccc737c8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:17.783104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7658bc-419d-4f02-a7d4-8275026f696b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:17.784610 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7658bc-419d-4f02-a7d4-8275026f696b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:17.799586 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:17.838208 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:17.880036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d579e-fd60-4d4e-b330-c546ccc737c8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:17.913303 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 186/363] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:17.914240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7658bc-419d-4f02-a7d4-8275026f696b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:17.930691 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8ae94-b829-469d-9ded-9cc47e84cc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:17.931645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8ae94-b829-469d-9ded-9cc47e84cc25 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:17.931923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8ae94-b829-469d-9ded-9cc47e84cc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:17.932195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8ae94-b829-469d-9ded-9cc47e84cc25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:17.944871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8ae94-b829-469d-9ded-9cc47e84cc25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-75f8ae94-b829-469d-9ded-9cc47e84cc25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:17.953209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:17.954106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:17.954931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:17.954931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:17.954931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:17.954931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:17.954931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:17.954931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:17.954931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:17.954931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:17.954931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:17.954931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:17.954931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:17.954931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:17.954931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 178/364] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:00:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:17.955641 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f47984-4677-4504-bdd4-1e60bd51166e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:17.956229 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f47984-4677-4504-bdd4-1e60bd51166e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:17.956229 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f47984-4677-4504-bdd4-1e60bd51166e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:17.975160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d9f47984-4677-4504-bdd4-1e60bd51166e None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:17.976371 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d9f47984-4677-4504-bdd4-1e60bd51166e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:18.041909 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7658bc-419d-4f02-a7d4-8275026f696b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:18.056522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:18.063319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 179/365] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:18.064053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f47984-4677-4504-bdd4-1e60bd51166e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:18.094615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797b132-3ba5-4071-8332-44200d9a926d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:18.095349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797b132-3ba5-4071-8332-44200d9a926d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:18.095636 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797b132-3ba5-4071-8332-44200d9a926d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:18.095913 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797b132-3ba5-4071-8332-44200d9a926d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:18.096772 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797b132-3ba5-4071-8332-44200d9a926d None admin] RBAC: Adding query filter params (group.id=d5308803b7ea4a8499209c30aa8d030d, role.id=b6dd71d4043d43e9823ec25176863347, scope.project.id=4d31d318708e4e9caa66106ce370a235) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:18.097088 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797b132-3ba5-4071-8332-44200d9a926d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:18.098548 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797b132-3ba5-4071-8332-44200d9a926d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:18.107793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 187/366] 10.222.0.67 () {56 vars in 1376 bytes} [Fri Jun 24 15:00:17 2022] GET /identity/v3/role_assignments?group.id=d5308803b7ea4a8499209c30aa8d030d&scope.project.id=4d31d318708e4e9caa66106ce370a235&role.id=b6dd71d4043d43e9823ec25176863347 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:18.107793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797b132-3ba5-4071-8332-44200d9a926d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:18.150321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:18.215438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f47984-4677-4504-bdd4-1e60bd51166e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:18.250230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797b132-3ba5-4071-8332-44200d9a926d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:18.264348 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7afbc-f63b-4da5-a741-6f6f6be042ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:18.266156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7afbc-f63b-4da5-a741-6f6f6be042ad None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:18.266614 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7afbc-f63b-4da5-a741-6f6f6be042ad None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:18.267029 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7afbc-f63b-4da5-a741-6f6f6be042ad None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:18.269280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7afbc-f63b-4da5-a741-6f6f6be042ad None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:18.271793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7afbc-f63b-4da5-a741-6f6f6be042ad None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:18.274849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 180/367] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:18 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:18.275872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7afbc-f63b-4da5-a741-6f6f6be042ad None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:18.301204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10968dfc-539e-4571-af5e-e5ed6d663719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:18.302514 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10968dfc-539e-4571-af5e-e5ed6d663719 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:18.302939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10968dfc-539e-4571-af5e-e5ed6d663719 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:18.303664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10968dfc-539e-4571-af5e-e5ed6d663719 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:18.304586 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10968dfc-539e-4571-af5e-e5ed6d663719 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:18.305034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10968dfc-539e-4571-af5e-e5ed6d663719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:18.306594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10968dfc-539e-4571-af5e-e5ed6d663719 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:18.316098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 188/368] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:18.317192 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10968dfc-539e-4571-af5e-e5ed6d663719 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:18.364228 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:18.407815 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7afbc-f63b-4da5-a741-6f6f6be042ad None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:18.440051 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:18.443896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10968dfc-539e-4571-af5e-e5ed6d663719 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:18.453258 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf9549-ff5e-4f71-92c4-0402dcba5273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:18.455030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf9549-ff5e-4f71-92c4-0402dcba5273 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:18.455571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf9549-ff5e-4f71-92c4-0402dcba5273 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:18.456048 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf9549-ff5e-4f71-92c4-0402dcba5273 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:18.472839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf9549-ff5e-4f71-92c4-0402dcba5273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-95cf9549-ff5e-4f71-92c4-0402dcba5273 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:18.485104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:18.486252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:18.487259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:18.487259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:18.487259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:18.487259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:18.487259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:18.487259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:18.487259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:18.487259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:18.487259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:18.487259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:18.487259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:00:18.487259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:18.488738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 181/369] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:00:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:18.491314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf9549-ff5e-4f71-92c4-0402dcba5273 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:18.495497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272c04d-df89-4d50-a3b3-c9de38bb6199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:18.496373 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272c04d-df89-4d50-a3b3-c9de38bb6199 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:18.496653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272c04d-df89-4d50-a3b3-c9de38bb6199 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:18.496942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272c04d-df89-4d50-a3b3-c9de38bb6199 None admin] PATH_INFO: `/v3/groups/d5308803b7ea4a8499209c30aa8d030d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:18.498078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272c04d-df89-4d50-a3b3-c9de38bb6199 None admin] RBAC: Authorizing `identity:get_group(group_id=d5308803b7ea4a8499209c30aa8d030d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:18.499564 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272c04d-df89-4d50-a3b3-c9de38bb6199 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:18.501168 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 189/370] 10.222.0.67 () {56 vars in 1180 bytes} [Fri Jun 24 15:00:18 2022] GET /identity/v3/groups/d5308803b7ea4a8499209c30aa8d030d => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:18.502431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272c04d-df89-4d50-a3b3-c9de38bb6199 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:18.637770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272c04d-df89-4d50-a3b3-c9de38bb6199 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:18.641020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf9549-ff5e-4f71-92c4-0402dcba5273 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:18.641308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:18.682798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725c7f0-2768-4a4d-b28e-f4026e8e2e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:18.684224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725c7f0-2768-4a4d-b28e-f4026e8e2e0b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:18.684224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725c7f0-2768-4a4d-b28e-f4026e8e2e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:18.684224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725c7f0-2768-4a4d-b28e-f4026e8e2e0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:18.696280 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ba35e-992d-44ca-b93d-cdb30cf3588d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:18.697004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ba35e-992d-44ca-b93d-cdb30cf3588d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:18.697355 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ba35e-992d-44ca-b93d-cdb30cf3588d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:18.697699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ba35e-992d-44ca-b93d-cdb30cf3588d None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:18.698520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ba35e-992d-44ca-b93d-cdb30cf3588d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:18.698912 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ba35e-992d-44ca-b93d-cdb30cf3588d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:18.700385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ba35e-992d-44ca-b93d-cdb30cf3588d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:18.710888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 190/371] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:18.710888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ba35e-992d-44ca-b93d-cdb30cf3588d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:18.756474 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:18.778467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725c7f0-2768-4a4d-b28e-f4026e8e2e0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:18.780230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725c7f0-2768-4a4d-b28e-f4026e8e2e0b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:18.839784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ba35e-992d-44ca-b93d-cdb30cf3588d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:18.882217 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f91ae5-b115-41a9-aff1-ff12a08949fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:18.883043 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f91ae5-b115-41a9-aff1-ff12a08949fb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:18.883462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f91ae5-b115-41a9-aff1-ff12a08949fb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:18.883836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f91ae5-b115-41a9-aff1-ff12a08949fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:18.885001 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f91ae5-b115-41a9-aff1-ff12a08949fb None admin] RBAC: Adding query filter params (group.id=d5308803b7ea4a8499209c30aa8d030d, role.id=b6dd71d4043d43e9823ec25176863347, scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:18.885446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f91ae5-b115-41a9-aff1-ff12a08949fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:18.887085 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f91ae5-b115-41a9-aff1-ff12a08949fb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:18.899208 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 191/372] 10.222.0.67 () {56 vars in 1376 bytes} [Fri Jun 24 15:00:18 2022] GET /identity/v3/role_assignments?group.id=d5308803b7ea4a8499209c30aa8d030d&scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487&role.id=b6dd71d4043d43e9823ec25176863347 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:18.899208 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f91ae5-b115-41a9-aff1-ff12a08949fb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:18.933780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 182/373] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:18.941056 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725c7f0-2768-4a4d-b28e-f4026e8e2e0b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:18.966776 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:19.090074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:19.110200 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725c7f0-2768-4a4d-b28e-f4026e8e2e0b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:19.121439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f91ae5-b115-41a9-aff1-ff12a08949fb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:19.148729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4966cd-1360-4a5c-91c3-d571ee2b5801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:19.149614 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4966cd-1360-4a5c-91c3-d571ee2b5801 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:19.149894 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4966cd-1360-4a5c-91c3-d571ee2b5801 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:19.150199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4966cd-1360-4a5c-91c3-d571ee2b5801 None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:19.151647 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4966cd-1360-4a5c-91c3-d571ee2b5801 None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:19.153054 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4966cd-1360-4a5c-91c3-d571ee2b5801 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:19.159168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 183/374] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:18 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:19.163023 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4966cd-1360-4a5c-91c3-d571ee2b5801 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:19.176755 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7b348-f233-45fa-81a7-f1a973eae86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:19.177739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7b348-f233-45fa-81a7-f1a973eae86f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:19.178483 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7b348-f233-45fa-81a7-f1a973eae86f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:19.179141 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7b348-f233-45fa-81a7-f1a973eae86f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:19.282082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7b348-f233-45fa-81a7-f1a973eae86f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:19.283877 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7b348-f233-45fa-81a7-f1a973eae86f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:19.297565 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4966cd-1360-4a5c-91c3-d571ee2b5801 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:19.322349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:19.343638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb970931-f095-42f1-b996-adbaccd69b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:19.344891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb970931-f095-42f1-b996-adbaccd69b46 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:19.345372 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb970931-f095-42f1-b996-adbaccd69b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:19.345873 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb970931-f095-42f1-b996-adbaccd69b46 None admin] PATH_INFO: `/v3/groups/d5308803b7ea4a8499209c30aa8d030d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:19.347290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb970931-f095-42f1-b996-adbaccd69b46 None admin] RBAC: Authorizing `identity:get_group(group_id=d5308803b7ea4a8499209c30aa8d030d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:19.349258 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb970931-f095-42f1-b996-adbaccd69b46 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:19.351748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 184/375] 10.222.0.67 () {56 vars in 1180 bytes} [Fri Jun 24 15:00:19 2022] GET /identity/v3/groups/d5308803b7ea4a8499209c30aa8d030d => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:19.358483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb970931-f095-42f1-b996-adbaccd69b46 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:19.396974 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:19.441402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 192/376] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:19.447434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7b348-f233-45fa-81a7-f1a973eae86f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:19.575202 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb970931-f095-42f1-b996-adbaccd69b46 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:19.600361 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7b348-f233-45fa-81a7-f1a973eae86f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:19.623469 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57e8a4-6f40-4953-9a03-297b69bdb22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:19.624137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57e8a4-6f40-4953-9a03-297b69bdb22a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:19.624422 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57e8a4-6f40-4953-9a03-297b69bdb22a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:19.624701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57e8a4-6f40-4953-9a03-297b69bdb22a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:19.625537 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57e8a4-6f40-4953-9a03-297b69bdb22a None admin] RBAC: Adding query filter params (group.id=d5308803b7ea4a8499209c30aa8d030d, role.id=c077ebdf7f214d7585dfc1570eecf882, scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:19.625851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57e8a4-6f40-4953-9a03-297b69bdb22a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:19.627306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57e8a4-6f40-4953-9a03-297b69bdb22a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:19.635394 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:19.636311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 185/377] 10.222.0.67 () {56 vars in 1376 bytes} [Fri Jun 24 15:00:19 2022] GET /identity/v3/role_assignments?group.id=d5308803b7ea4a8499209c30aa8d030d&scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487&role.id=c077ebdf7f214d7585dfc1570eecf882 => generated 260 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:19.642330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5eefb-f6b6-44d5-a201-e87607ec04c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:19.643241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5eefb-f6b6-44d5-a201-e87607ec04c3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:19.643530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5eefb-f6b6-44d5-a201-e87607ec04c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:19.646814 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5eefb-f6b6-44d5-a201-e87607ec04c3 None admin] PATH_INFO: `/v3/projects/4e36bcd54fa5438c93e449d16bda5ca3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:19.646917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850f6e9-a943-4ead-a04b-40a950570d91 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:19.646917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850f6e9-a943-4ead-a04b-40a950570d91 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:19.647053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5eefb-f6b6-44d5-a201-e87607ec04c3 None admin] RBAC: Authorizing `identity:get_project(project_id=4e36bcd54fa5438c93e449d16bda5ca3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:19.647121 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850f6e9-a943-4ead-a04b-40a950570d91 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:19.647358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5eefb-f6b6-44d5-a201-e87607ec04c3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:19.647699 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 186/378] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:19 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:19.649718 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 193/379] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:19 2022] GET /identity/v3/projects/4e36bcd54fa5438c93e449d16bda5ca3 => generated 344 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:19.653608 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25420e26-9543-4b73-8fa1-aea5efa10ce3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:19.653903 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25420e26-9543-4b73-8fa1-aea5efa10ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:19.654184 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25420e26-9543-4b73-8fa1-aea5efa10ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:19.659022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5eefb-f6b6-44d5-a201-e87607ec04c3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:19.676990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-25420e26-9543-4b73-8fa1-aea5efa10ce3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:19.679318 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-25420e26-9543-4b73-8fa1-aea5efa10ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:19.768494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 187/380] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:19.846966 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5eefb-f6b6-44d5-a201-e87607ec04c3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:19.863277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25420e26-9543-4b73-8fa1-aea5efa10ce3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:19.885591 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:19.903981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebb642-b4a9-4333-84cc-748b8189bba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:19.904867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebb642-b4a9-4333-84cc-748b8189bba5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:19.905159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebb642-b4a9-4333-84cc-748b8189bba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:19.906843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebb642-b4a9-4333-84cc-748b8189bba5 None admin] PATH_INFO: `/v3/groups/9e059c031abd45e2a952d8771961b440` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:19.912166 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebb642-b4a9-4333-84cc-748b8189bba5 None admin] RBAC: Authorizing `identity:get_group(group_id=9e059c031abd45e2a952d8771961b440)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:19.913974 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebb642-b4a9-4333-84cc-748b8189bba5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:19.921082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 194/381] 10.222.0.67 () {56 vars in 1180 bytes} [Fri Jun 24 15:00:19 2022] GET /identity/v3/groups/9e059c031abd45e2a952d8771961b440 => generated 229 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:19.930305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebb642-b4a9-4333-84cc-748b8189bba5 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:20.035626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:20.056337 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25420e26-9543-4b73-8fa1-aea5efa10ce3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:20.109010 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebb642-b4a9-4333-84cc-748b8189bba5 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:20.128728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5b94b-3bc7-4190-a94a-ca2412fa1c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:20.129615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5b94b-3bc7-4190-a94a-ca2412fa1c6e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:20.129906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5b94b-3bc7-4190-a94a-ca2412fa1c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:20.130187 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5b94b-3bc7-4190-a94a-ca2412fa1c6e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:20.152714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5b94b-3bc7-4190-a94a-ca2412fa1c6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-e9c5b94b-3bc7-4190-a94a-ca2412fa1c6e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:20.165324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:20.166255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:20.167069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:20.167069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:20.167069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:20.167069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:20.167069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:20.167069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:20.167069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:20.167069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:20.167069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:20.167069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:20.167069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:20.167069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:20.169949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 188/382] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:00:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:20.175986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5b94b-3bc7-4190-a94a-ca2412fa1c6e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:20.196484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f33d2-1c78-4f0a-8e23-2b7e0042129a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:20.197171 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f33d2-1c78-4f0a-8e23-2b7e0042129a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:20.197456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f33d2-1c78-4f0a-8e23-2b7e0042129a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:20.197728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f33d2-1c78-4f0a-8e23-2b7e0042129a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:20.198562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f33d2-1c78-4f0a-8e23-2b7e0042129a None admin] RBAC: Adding query filter params (group.id=9e059c031abd45e2a952d8771961b440, role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=4e36bcd54fa5438c93e449d16bda5ca3) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:20.198872 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f33d2-1c78-4f0a-8e23-2b7e0042129a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:20.200413 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f33d2-1c78-4f0a-8e23-2b7e0042129a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:20.221126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 195/383] 10.222.0.67 () {56 vars in 1376 bytes} [Fri Jun 24 15:00:19 2022] GET /identity/v3/role_assignments?group.id=9e059c031abd45e2a952d8771961b440&scope.project.id=4e36bcd54fa5438c93e449d16bda5ca3&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 260 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:20.275595 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:20.299947 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5b94b-3bc7-4190-a94a-ca2412fa1c6e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:20.342888 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fcf93-816e-4d30-a6d6-047290c1b699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:20.343621 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fcf93-816e-4d30-a6d6-047290c1b699 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:20.343913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fcf93-816e-4d30-a6d6-047290c1b699 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:20.344194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fcf93-816e-4d30-a6d6-047290c1b699 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:20.344969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8fcf93-816e-4d30-a6d6-047290c1b699 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:20.345285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8fcf93-816e-4d30-a6d6-047290c1b699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:20.346717 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8fcf93-816e-4d30-a6d6-047290c1b699 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:20.359307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 189/384] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:20.365487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f33d2-1c78-4f0a-8e23-2b7e0042129a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:20.503923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f33d2-1c78-4f0a-8e23-2b7e0042129a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:20.530761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:20.540824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ce558-d8dd-4b38-9ba2-e865c52bab28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:20.541689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ce558-d8dd-4b38-9ba2-e865c52bab28 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:20.541969 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ce558-d8dd-4b38-9ba2-e865c52bab28 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:20.542241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ce558-d8dd-4b38-9ba2-e865c52bab28 None admin] PATH_INFO: `/v3/groups/d5308803b7ea4a8499209c30aa8d030d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:20.545337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ce558-d8dd-4b38-9ba2-e865c52bab28 None admin] RBAC: Authorizing `identity:get_group(group_id=d5308803b7ea4a8499209c30aa8d030d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:20.546791 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ce558-d8dd-4b38-9ba2-e865c52bab28 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:20.548854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 196/385] 10.222.0.67 () {56 vars in 1180 bytes} [Fri Jun 24 15:00:20 2022] GET /identity/v3/groups/d5308803b7ea4a8499209c30aa8d030d => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:20.551567 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d8a4c-84a0-43ca-8dfc-07c9ee0b586b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:20.551980 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d8a4c-84a0-43ca-8dfc-07c9ee0b586b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:20.552888 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d8a4c-84a0-43ca-8dfc-07c9ee0b586b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:20.554926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ce558-d8dd-4b38-9ba2-e865c52bab28 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:20.561843 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 190/386] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:20 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:20.563317 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e72e56-abea-42d9-90c2-7e77d3ff6d11 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:20.563716 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e72e56-abea-42d9-90c2-7e77d3ff6d11 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:20.564090 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e72e56-abea-42d9-90c2-7e77d3ff6d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:20.585372 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-26e72e56-abea-42d9-90c2-7e77d3ff6d11 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:20.586983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-26e72e56-abea-42d9-90c2-7e77d3ff6d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:20.682975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 191/387] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:20.709569 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ce558-d8dd-4b38-9ba2-e865c52bab28 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:20.710646 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e72e56-abea-42d9-90c2-7e77d3ff6d11 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:20.768137 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f67c0-a447-4818-95e1-3dcef0da2fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:20.769234 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f67c0-a447-4818-95e1-3dcef0da2fde None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:20.769663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f67c0-a447-4818-95e1-3dcef0da2fde None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:20.770098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f67c0-a447-4818-95e1-3dcef0da2fde None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:20.773926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f67c0-a447-4818-95e1-3dcef0da2fde None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:20.775410 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f67c0-a447-4818-95e1-3dcef0da2fde None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:20.777843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 197/388] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:20 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:20.783989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f67c0-a447-4818-95e1-3dcef0da2fde None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:20.785729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:20.829277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e72e56-abea-42d9-90c2-7e77d3ff6d11 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:20.872473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cbea2-8c19-433f-b4c6-67128e9347fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:20.874090 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cbea2-8c19-433f-b4c6-67128e9347fd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:20.874961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cbea2-8c19-433f-b4c6-67128e9347fd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:20.875475 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cbea2-8c19-433f-b4c6-67128e9347fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:20.886821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715cbea2-8c19-433f-b4c6-67128e9347fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-715cbea2-8c19-433f-b4c6-67128e9347fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:20.900921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:20.901912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:20.902710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:20.902710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:20.902710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:20.902710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:20.902710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:20.902710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:20.902710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:20.902710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:20.902710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:20.902710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:20.902710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:20.902710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:20.905439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 192/389] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:20.912245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cbea2-8c19-433f-b4c6-67128e9347fd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:20.916106 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:21.001313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f67c0-a447-4818-95e1-3dcef0da2fde None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:21.033851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cbea2-8c19-433f-b4c6-67128e9347fd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:21.067654 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:21.084505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43dbcf-d6c3-49cf-8478-413f408180de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:21.096987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43dbcf-d6c3-49cf-8478-413f408180de None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:21.097369 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bc3b1-7600-4c31-a3b4-466fec1aad88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:21.098549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bc3b1-7600-4c31-a3b4-466fec1aad88 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:21.099542 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43dbcf-d6c3-49cf-8478-413f408180de None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:21.100019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bc3b1-7600-4c31-a3b4-466fec1aad88 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:21.100528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43dbcf-d6c3-49cf-8478-413f408180de None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:21.100992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bc3b1-7600-4c31-a3b4-466fec1aad88 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235/groups/d5308803b7ea4a8499209c30aa8d030d/roles/c077ebdf7f214d7585dfc1570eecf882` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:21.102743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43dbcf-d6c3-49cf-8478-413f408180de None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:21.103833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43dbcf-d6c3-49cf-8478-413f408180de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:21.105438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43dbcf-d6c3-49cf-8478-413f408180de None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:21.106866 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bc3b1-7600-4c31-a3b4-466fec1aad88 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d31d318708e4e9caa66106ce370a235, group_id=d5308803b7ea4a8499209c30aa8d030d, role_id=c077ebdf7f214d7585dfc1570eecf882)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:21.110147 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bc3b1-7600-4c31-a3b4-466fec1aad88 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:21.121908 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 193/390] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:21.129080 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43dbcf-d6c3-49cf-8478-413f408180de None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:21.209786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 198/391] 10.222.0.67 () {58 vars in 1442 bytes} [Fri Jun 24 15:00:20 2022] PUT /identity/v3/projects/4d31d318708e4e9caa66106ce370a235/groups/d5308803b7ea4a8499209c30aa8d030d/roles/c077ebdf7f214d7585dfc1570eecf882 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:00:21.248084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43dbcf-d6c3-49cf-8478-413f408180de None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:21.294838 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b50f7f-5c1a-4aeb-8a26-5e06360c00d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:21.295686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b50f7f-5c1a-4aeb-8a26-5e06360c00d7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:21.295972 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b50f7f-5c1a-4aeb-8a26-5e06360c00d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:21.296249 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b50f7f-5c1a-4aeb-8a26-5e06360c00d7 None admin] PATH_INFO: `/v3/groups/d5308803b7ea4a8499209c30aa8d030d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:21.297437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b50f7f-5c1a-4aeb-8a26-5e06360c00d7 None admin] RBAC: Authorizing `identity:get_group(group_id=d5308803b7ea4a8499209c30aa8d030d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:21.298884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b50f7f-5c1a-4aeb-8a26-5e06360c00d7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:21.300549 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 194/392] 10.222.0.67 () {56 vars in 1180 bytes} [Fri Jun 24 15:00:21 2022] GET /identity/v3/groups/d5308803b7ea4a8499209c30aa8d030d => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:21.307360 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bc3b1-7600-4c31-a3b4-466fec1aad88 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:21.462956 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bc3b1-7600-4c31-a3b4-466fec1aad88 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:21.490837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:21.503774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69852983-5b43-4099-b12c-df43150aab7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:21.505385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69852983-5b43-4099-b12c-df43150aab7a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:21.506083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69852983-5b43-4099-b12c-df43150aab7a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:21.506520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69852983-5b43-4099-b12c-df43150aab7a None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:21.507929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69852983-5b43-4099-b12c-df43150aab7a None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:21.509528 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69852983-5b43-4099-b12c-df43150aab7a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:21.511741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 199/393] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:21 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:21.518067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b50f7f-5c1a-4aeb-8a26-5e06360c00d7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:21.535879 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c41bb-c08e-446a-bbd4-77df2dfdd73e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:21.536187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c41bb-c08e-446a-bbd4-77df2dfdd73e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:21.536434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c41bb-c08e-446a-bbd4-77df2dfdd73e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:21.538410 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 200/394] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:21.543634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a22e0-2db2-4fce-ba5e-0045d2efa4c7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:21.543919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a22e0-2db2-4fce-ba5e-0045d2efa4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:21.544208 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a22e0-2db2-4fce-ba5e-0045d2efa4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:21.562162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-548a22e0-2db2-4fce-ba5e-0045d2efa4c7 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:21.563314 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-548a22e0-2db2-4fce-ba5e-0045d2efa4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:21.601843 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:21.651257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 201/395] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:21.685183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b50f7f-5c1a-4aeb-8a26-5e06360c00d7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:21.691438 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a22e0-2db2-4fce-ba5e-0045d2efa4c7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:21.736933 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa35a3-61d2-4432-8f4c-676bdf3b2265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:21.737833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa35a3-61d2-4432-8f4c-676bdf3b2265 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:21.738120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa35a3-61d2-4432-8f4c-676bdf3b2265 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:21.738402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa35a3-61d2-4432-8f4c-676bdf3b2265 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235/groups/d5308803b7ea4a8499209c30aa8d030d/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:21.743007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa35a3-61d2-4432-8f4c-676bdf3b2265 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d31d318708e4e9caa66106ce370a235, group_id=d5308803b7ea4a8499209c30aa8d030d, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:21.744504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa35a3-61d2-4432-8f4c-676bdf3b2265 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:21.841870 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a22e0-2db2-4fce-ba5e-0045d2efa4c7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:21.863695 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:21.879024 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4236a-f227-4fa0-9ca4-07a392ece9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:21.879860 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4236a-f227-4fa0-9ca4-07a392ece9bc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:21.880140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4236a-f227-4fa0-9ca4-07a392ece9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:21.880927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4236a-f227-4fa0-9ca4-07a392ece9bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:21.893975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4236a-f227-4fa0-9ca4-07a392ece9bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f9e4236a-f227-4fa0-9ca4-07a392ece9bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:21.903244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:21.904325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:21.905659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:21.905659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:21.905659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:21.905659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:21.905659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:21.905659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:21.905659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:21.905659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:21.905659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:21.905659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:21.905659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:21.905659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:21.905659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 202/396] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:21.913546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4236a-f227-4fa0-9ca4-07a392ece9bc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:22.024130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4236a-f227-4fa0-9ca4-07a392ece9bc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:22.060188 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff1444-853c-41d4-9dde-446594e67a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:22.060859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff1444-853c-41d4-9dde-446594e67a75 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:22.061141 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff1444-853c-41d4-9dde-446594e67a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:22.061420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff1444-853c-41d4-9dde-446594e67a75 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:22.062171 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff1444-853c-41d4-9dde-446594e67a75 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:22.062475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff1444-853c-41d4-9dde-446594e67a75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:22.063871 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff1444-853c-41d4-9dde-446594e67a75 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:22.072854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 203/397] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:22.080240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff1444-853c-41d4-9dde-446594e67a75 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:22.124414 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 195/398] 10.222.0.67 () {58 vars in 1442 bytes} [Fri Jun 24 15:00:21 2022] PUT /identity/v3/projects/4d31d318708e4e9caa66106ce370a235/groups/d5308803b7ea4a8499209c30aa8d030d/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:00:22.137522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:22.201596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff1444-853c-41d4-9dde-446594e67a75 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:22.237751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb5a7d-7558-43bb-9645-b49fffd19950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:22.238621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb5a7d-7558-43bb-9645-b49fffd19950 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:22.238901 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb5a7d-7558-43bb-9645-b49fffd19950 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:22.239188 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb5a7d-7558-43bb-9645-b49fffd19950 None admin] PATH_INFO: `/v3/groups/d5308803b7ea4a8499209c30aa8d030d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:22.240961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb5a7d-7558-43bb-9645-b49fffd19950 None admin] RBAC: Authorizing `identity:get_group(group_id=d5308803b7ea4a8499209c30aa8d030d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:22.242497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb5a7d-7558-43bb-9645-b49fffd19950 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:22.244490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 204/399] 10.222.0.67 () {56 vars in 1180 bytes} [Fri Jun 24 15:00:22 2022] GET /identity/v3/groups/d5308803b7ea4a8499209c30aa8d030d => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:22.249973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa35a3-61d2-4432-8f4c-676bdf3b2265 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:22.377378 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa35a3-61d2-4432-8f4c-676bdf3b2265 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:22.403276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:22.421839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ec862-b36d-4c3b-8194-d443c9a8ea77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:22.428836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ec862-b36d-4c3b-8194-d443c9a8ea77 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:22.428836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ec862-b36d-4c3b-8194-d443c9a8ea77 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:22.428836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ec862-b36d-4c3b-8194-d443c9a8ea77 None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:22.428836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ec862-b36d-4c3b-8194-d443c9a8ea77 None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:22.428836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ec862-b36d-4c3b-8194-d443c9a8ea77 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:22.428836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 196/400] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:22 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:22.433735 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb5a7d-7558-43bb-9645-b49fffd19950 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:22.543124 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb5a7d-7558-43bb-9645-b49fffd19950 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:22.556778 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:22.579191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed4544-902d-4631-b254-e19e4c881589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:22.580088 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed4544-902d-4631-b254-e19e4c881589 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:22.580368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed4544-902d-4631-b254-e19e4c881589 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:22.580642 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed4544-902d-4631-b254-e19e4c881589 None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487/groups/d5308803b7ea4a8499209c30aa8d030d/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:22.582694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed4544-902d-4631-b254-e19e4c881589 None admin] RBAC: Authorizing `identity:create_grant(project_id=01d2a11c551c4d8fbf8eb173cd1cc487, group_id=d5308803b7ea4a8499209c30aa8d030d, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:22.584992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed4544-902d-4631-b254-e19e4c881589 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:22.622294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d580a78-fb3b-49e3-b521-cd87e578012d None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:22.622637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d580a78-fb3b-49e3-b521-cd87e578012d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:22.622853 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d580a78-fb3b-49e3-b521-cd87e578012d None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:22.624551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 197/401] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:22.630063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeffc41-70b4-4879-a524-33f0cd67f59c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:22.630855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeffc41-70b4-4879-a524-33f0cd67f59c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:22.631238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeffc41-70b4-4879-a524-33f0cd67f59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:22.656541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 205/402] 10.222.0.67 () {58 vars in 1442 bytes} [Fri Jun 24 15:00:22 2022] PUT /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487/groups/d5308803b7ea4a8499209c30aa8d030d/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:00:22.656757 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8aeffc41-70b4-4879-a524-33f0cd67f59c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:22.657681 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8aeffc41-70b4-4879-a524-33f0cd67f59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:22.744714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 198/403] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:22.793270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed4544-902d-4631-b254-e19e4c881589 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:22.931493 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed4544-902d-4631-b254-e19e4c881589 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:22.969018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a447c6b-3427-47ee-9823-6eb40a1fe025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:22.969843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a447c6b-3427-47ee-9823-6eb40a1fe025 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:22.970125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a447c6b-3427-47ee-9823-6eb40a1fe025 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:22.970399 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a447c6b-3427-47ee-9823-6eb40a1fe025 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:22.986108 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a447c6b-3427-47ee-9823-6eb40a1fe025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9a447c6b-3427-47ee-9823-6eb40a1fe025 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:22.994859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:22.995783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:22.996598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:22.996598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:22.996598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:22.996598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:22.996598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:22.996598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:22.996598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:22.996598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:22.996598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:22.996598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:22.996598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:22.996598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:22.996598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 206/404] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:00:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:23.002284 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeffc41-70b4-4879-a524-33f0cd67f59c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:23.067034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad36b1d-c078-4412-bbab-c5862e8bc868 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:23.067372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad36b1d-c078-4412-bbab-c5862e8bc868 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:23.067644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad36b1d-c078-4412-bbab-c5862e8bc868 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:23.070040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 207/405] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:23.081037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54dbf-9439-45ff-bb51-62a1a94965f4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:23.081326 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54dbf-9439-45ff-bb51-62a1a94965f4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:23.081603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54dbf-9439-45ff-bb51-62a1a94965f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:23.123363 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeffc41-70b4-4879-a524-33f0cd67f59c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:23.124266 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9cf54dbf-9439-45ff-bb51-62a1a94965f4 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:23.125464 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9cf54dbf-9439-45ff-bb51-62a1a94965f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:23.138930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:23.171072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:23.172553 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90144b0-9f90-4542-a31b-d6f654511f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:23.173245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90144b0-9f90-4542-a31b-d6f654511f4f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:23.173528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90144b0-9f90-4542-a31b-d6f654511f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:23.173806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90144b0-9f90-4542-a31b-d6f654511f4f None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:23.174645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90144b0-9f90-4542-a31b-d6f654511f4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:23.174946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90144b0-9f90-4542-a31b-d6f654511f4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:23.176621 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90144b0-9f90-4542-a31b-d6f654511f4f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:23.192405 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 199/406] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:23.198051 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90144b0-9f90-4542-a31b-d6f654511f4f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:23.261919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 208/407] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 15:00:23.330778 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54dbf-9439-45ff-bb51-62a1a94965f4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:23.355089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90144b0-9f90-4542-a31b-d6f654511f4f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:23.409752 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b056cb2-fb6c-43f4-9070-b30a4584bef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:23.410630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b056cb2-fb6c-43f4-9070-b30a4584bef1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:23.410915 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b056cb2-fb6c-43f4-9070-b30a4584bef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:23.411213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b056cb2-fb6c-43f4-9070-b30a4584bef1 None admin] PATH_INFO: `/v3/groups/d5308803b7ea4a8499209c30aa8d030d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:23.414087 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b056cb2-fb6c-43f4-9070-b30a4584bef1 None admin] RBAC: Authorizing `identity:get_group(group_id=d5308803b7ea4a8499209c30aa8d030d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:23.415563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b056cb2-fb6c-43f4-9070-b30a4584bef1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:23.420028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 200/408] 10.222.0.67 () {56 vars in 1180 bytes} [Fri Jun 24 15:00:23 2022] GET /identity/v3/groups/d5308803b7ea4a8499209c30aa8d030d => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:23.425096 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b056cb2-fb6c-43f4-9070-b30a4584bef1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:23.515613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:23.521613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54dbf-9439-45ff-bb51-62a1a94965f4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:23.594045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0bfbf-1bfd-46ac-8f0f-86d6788158db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:23.594864 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0bfbf-1bfd-46ac-8f0f-86d6788158db None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:23.595151 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0bfbf-1bfd-46ac-8f0f-86d6788158db None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:23.595453 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0bfbf-1bfd-46ac-8f0f-86d6788158db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:23.614265 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b056cb2-fb6c-43f4-9070-b30a4584bef1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:23.619248 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0bfbf-1bfd-46ac-8f0f-86d6788158db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-1bb0bfbf-1bfd-46ac-8f0f-86d6788158db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:23.627123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:23.628071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:23.628981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:23.628981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:23.628981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:23.628981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:23.628981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:23.628981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:23.628981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:23.628981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:23.628981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:23.628981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:23.628981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:00:23.628981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:23.636421 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 209/409] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:00:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:23.644344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0bfbf-1bfd-46ac-8f0f-86d6788158db None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:23.656272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2b55e-783b-4c27-86e9-3033e2a58cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:23.657380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2b55e-783b-4c27-86e9-3033e2a58cca None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:23.657868 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2b55e-783b-4c27-86e9-3033e2a58cca None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:23.658319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2b55e-783b-4c27-86e9-3033e2a58cca None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:23.659928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2b55e-783b-4c27-86e9-3033e2a58cca None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:23.662340 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2b55e-783b-4c27-86e9-3033e2a58cca None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:23.664943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 201/410] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:23 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:23.671381 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2b55e-783b-4c27-86e9-3033e2a58cca None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:23.703548 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:23.810341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2b55e-783b-4c27-86e9-3033e2a58cca None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:23.827588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:23.848365 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe4792-aa21-4e3c-8384-574b4a7a45b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:23.850298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe4792-aa21-4e3c-8384-574b4a7a45b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:23.850708 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe4792-aa21-4e3c-8384-574b4a7a45b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:23.850993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe4792-aa21-4e3c-8384-574b4a7a45b8 None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487/groups/d5308803b7ea4a8499209c30aa8d030d/roles/c077ebdf7f214d7585dfc1570eecf882` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:23.853287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe4792-aa21-4e3c-8384-574b4a7a45b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=01d2a11c551c4d8fbf8eb173cd1cc487, group_id=d5308803b7ea4a8499209c30aa8d030d, role_id=c077ebdf7f214d7585dfc1570eecf882)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:23.854740 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe4792-aa21-4e3c-8384-574b4a7a45b8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:23.890376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0bfbf-1bfd-46ac-8f0f-86d6788158db None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:23.924817 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 202/411] 10.222.0.67 () {58 vars in 1442 bytes} [Fri Jun 24 15:00:23 2022] PUT /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487/groups/d5308803b7ea4a8499209c30aa8d030d/roles/c077ebdf7f214d7585dfc1570eecf882 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:00:23.926734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49858ab0-7141-45b1-9af5-6b47a392593a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:23.927784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49858ab0-7141-45b1-9af5-6b47a392593a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:23.927904 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461ebd1-7356-4251-8198-742bc9609a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:23.928625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49858ab0-7141-45b1-9af5-6b47a392593a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:23.929055 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461ebd1-7356-4251-8198-742bc9609a65 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:23.929443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461ebd1-7356-4251-8198-742bc9609a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:23.929830 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461ebd1-7356-4251-8198-742bc9609a65 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:23.930669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461ebd1-7356-4251-8198-742bc9609a65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:23.931056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461ebd1-7356-4251-8198-742bc9609a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:23.935421 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461ebd1-7356-4251-8198-742bc9609a65 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:23.936033 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 203/412] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:23 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:23.938286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e8aa9-2566-4a7f-a5dc-b48a43465dbd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:23.938767 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e8aa9-2566-4a7f-a5dc-b48a43465dbd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:23.939245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e8aa9-2566-4a7f-a5dc-b48a43465dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:23.944146 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 210/413] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:23.949917 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461ebd1-7356-4251-8198-742bc9609a65 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:23.961921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-239e8aa9-2566-4a7f-a5dc-b48a43465dbd None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:23.963953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-239e8aa9-2566-4a7f-a5dc-b48a43465dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:24.057352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 204/414] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:24.087630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e8aa9-2566-4a7f-a5dc-b48a43465dbd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:24.113453 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:24.148683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461ebd1-7356-4251-8198-742bc9609a65 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:24.215428 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e8aa9-2566-4a7f-a5dc-b48a43465dbd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:24.217882 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:24.223863 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38db049-2b68-47ca-9270-0a92f11c4a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:24.225270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38db049-2b68-47ca-9270-0a92f11c4a3b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:24.225751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38db049-2b68-47ca-9270-0a92f11c4a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:24.226030 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38db049-2b68-47ca-9270-0a92f11c4a3b None admin] PATH_INFO: `/v3/groups/9e059c031abd45e2a952d8771961b440` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:24.228157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38db049-2b68-47ca-9270-0a92f11c4a3b None admin] RBAC: Authorizing `identity:get_group(group_id=9e059c031abd45e2a952d8771961b440)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:24.230005 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38db049-2b68-47ca-9270-0a92f11c4a3b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:24.231792 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 211/415] 10.222.0.67 () {56 vars in 1180 bytes} [Fri Jun 24 15:00:23 2022] GET /identity/v3/groups/9e059c031abd45e2a952d8771961b440 => generated 229 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:24.237456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38db049-2b68-47ca-9270-0a92f11c4a3b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:24.259604 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40458512-2fb7-44cf-8c59-3413280e2064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:24.264773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40458512-2fb7-44cf-8c59-3413280e2064 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:24.264773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40458512-2fb7-44cf-8c59-3413280e2064 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:24.264773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40458512-2fb7-44cf-8c59-3413280e2064 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:24.276753 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40458512-2fb7-44cf-8c59-3413280e2064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-40458512-2fb7-44cf-8c59-3413280e2064 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:24.289482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:24.290415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:24.291268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:24.291268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:24.291268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:24.291268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:24.291268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:24.291268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:24.291268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:24.291268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:24.291268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:24.291268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:24.291268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:24.291268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:24.293154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 205/416] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:00:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:24.300087 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40458512-2fb7-44cf-8c59-3413280e2064 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:24.406065 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38db049-2b68-47ca-9270-0a92f11c4a3b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:24.470443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f2d69-289d-4e19-a3dd-dbb375825144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:24.471337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f2d69-289d-4e19-a3dd-dbb375825144 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:24.471618 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f2d69-289d-4e19-a3dd-dbb375825144 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:24.471890 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f2d69-289d-4e19-a3dd-dbb375825144 None admin] PATH_INFO: `/v3/projects/4e36bcd54fa5438c93e449d16bda5ca3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:24.473777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f2d69-289d-4e19-a3dd-dbb375825144 None admin] RBAC: Authorizing `identity:get_project(project_id=4e36bcd54fa5438c93e449d16bda5ca3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:24.475941 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f2d69-289d-4e19-a3dd-dbb375825144 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:24.478738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 212/417] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:24 2022] GET /identity/v3/projects/4e36bcd54fa5438c93e449d16bda5ca3 => generated 344 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:24.485300 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40458512-2fb7-44cf-8c59-3413280e2064 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:24.487862 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f2d69-289d-4e19-a3dd-dbb375825144 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:24.535547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:24.549705 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0383b-4a25-48a2-833b-112d9aa0e380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:24.550384 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0383b-4a25-48a2-833b-112d9aa0e380 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:24.550669 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0383b-4a25-48a2-833b-112d9aa0e380 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:24.550946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0383b-4a25-48a2-833b-112d9aa0e380 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:24.551763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0383b-4a25-48a2-833b-112d9aa0e380 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:24.552593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0383b-4a25-48a2-833b-112d9aa0e380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:24.554447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0383b-4a25-48a2-833b-112d9aa0e380 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:24.563466 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 206/418] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:24.569773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0383b-4a25-48a2-833b-112d9aa0e380 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:24.603427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:24.684830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0383b-4a25-48a2-833b-112d9aa0e380 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:24.731247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f2d69-289d-4e19-a3dd-dbb375825144 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:24.738452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede39df5-d101-4645-ae15-eed4f8780773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:24.738452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede39df5-d101-4645-ae15-eed4f8780773 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:24.738452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede39df5-d101-4645-ae15-eed4f8780773 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:24.738452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede39df5-d101-4645-ae15-eed4f8780773 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:24.801638 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c8cfe-eefa-49cc-8128-f46a8825397e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:24.803597 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c8cfe-eefa-49cc-8128-f46a8825397e None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:24.803903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c8cfe-eefa-49cc-8128-f46a8825397e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:24.804143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c8cfe-eefa-49cc-8128-f46a8825397e None admin] PATH_INFO: `/v3/projects/4e36bcd54fa5438c93e449d16bda5ca3/groups/9e059c031abd45e2a952d8771961b440/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:24.807064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c8cfe-eefa-49cc-8128-f46a8825397e None admin] RBAC: Authorizing `identity:create_grant(project_id=4e36bcd54fa5438c93e449d16bda5ca3, group_id=9e059c031abd45e2a952d8771961b440, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:24.808521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c8cfe-eefa-49cc-8128-f46a8825397e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:24.842527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede39df5-d101-4645-ae15-eed4f8780773 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:24.844446 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede39df5-d101-4645-ae15-eed4f8780773 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:24.871859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:24.886239 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 213/419] 10.222.0.67 () {58 vars in 1442 bytes} [Fri Jun 24 15:00:24 2022] PUT /identity/v3/projects/4e36bcd54fa5438c93e449d16bda5ca3/groups/9e059c031abd45e2a952d8771961b440/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:00:24.888400 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef7ef9-71ea-463f-a444-cb7d945b13c2 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:24.888899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef7ef9-71ea-463f-a444-cb7d945b13c2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:24.889370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef7ef9-71ea-463f-a444-cb7d945b13c2 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:24.892014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 214/420] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:24.899802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72862f9-db1d-4a1c-839e-a9a1825bdb58 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:24.900417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72862f9-db1d-4a1c-839e-a9a1825bdb58 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:24.900885 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72862f9-db1d-4a1c-839e-a9a1825bdb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:24.922750 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f72862f9-db1d-4a1c-839e-a9a1825bdb58 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:24.924418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f72862f9-db1d-4a1c-839e-a9a1825bdb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:24.966228 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 207/421] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:24.981004 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede39df5-d101-4645-ae15-eed4f8780773 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:25.029831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 215/422] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:25.070166 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72862f9-db1d-4a1c-839e-a9a1825bdb58 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:25.081014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:25.141679 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede39df5-d101-4645-ae15-eed4f8780773 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:25.170853 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:25.185391 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a10ade-38db-449a-bca7-cfa34be032fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:25.187064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a10ade-38db-449a-bca7-cfa34be032fc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:25.187377 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a10ade-38db-449a-bca7-cfa34be032fc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:25.187657 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a10ade-38db-449a-bca7-cfa34be032fc None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:25.188863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a10ade-38db-449a-bca7-cfa34be032fc None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:25.190854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a10ade-38db-449a-bca7-cfa34be032fc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:25.194420 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 208/423] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:24 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:25.195686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911dec0-8d84-4a29-b4e0-4c4626e4752e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:25.195969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911dec0-8d84-4a29-b4e0-4c4626e4752e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:25.196243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911dec0-8d84-4a29-b4e0-4c4626e4752e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:25.197486 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 209/424] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:25.200091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911dec0-8d84-4a29-b4e0-4c4626e4752e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:25.210822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72862f9-db1d-4a1c-839e-a9a1825bdb58 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:25.253310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b1571-8f4e-46a4-8e63-d4617560276f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:25.254049 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b1571-8f4e-46a4-8e63-d4617560276f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:25.254372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b1571-8f4e-46a4-8e63-d4617560276f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:25.254682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b1571-8f4e-46a4-8e63-d4617560276f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:25.269866 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b1571-8f4e-46a4-8e63-d4617560276f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-810b1571-8f4e-46a4-8e63-d4617560276f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:25.279806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:25.280712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:25.281731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:25.281731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:25.281731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:25.281731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:25.281731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:25.281731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:25.281731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:25.281731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:25.281731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:25.281731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:25.281731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:25.281731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:25.281731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 216/425] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:25.283581 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0654740-fc47-4638-83b5-0b2357d29cff None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:25.283581 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0654740-fc47-4638-83b5-0b2357d29cff None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:25.283784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0654740-fc47-4638-83b5-0b2357d29cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:25.305570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e0654740-fc47-4638-83b5-0b2357d29cff None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:25.307211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e0654740-fc47-4638-83b5-0b2357d29cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:25.331632 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911dec0-8d84-4a29-b4e0-4c4626e4752e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:25.374366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe9c9b-a5a9-439a-93d4-6ba1de7cd3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:25.375809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe9c9b-a5a9-439a-93d4-6ba1de7cd3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:25.376098 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe9c9b-a5a9-439a-93d4-6ba1de7cd3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:25.376647 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe9c9b-a5a9-439a-93d4-6ba1de7cd3e2 None admin] PATH_INFO: `/v3/groups/d5308803b7ea4a8499209c30aa8d030d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:25.378085 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe9c9b-a5a9-439a-93d4-6ba1de7cd3e2 None admin] RBAC: Authorizing `identity:get_group(group_id=d5308803b7ea4a8499209c30aa8d030d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:25.380802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe9c9b-a5a9-439a-93d4-6ba1de7cd3e2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:25.383653 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 210/426] 10.222.0.67 () {56 vars in 1180 bytes} [Fri Jun 24 15:00:25 2022] GET /identity/v3/groups/d5308803b7ea4a8499209c30aa8d030d => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:25.384438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe9c9b-a5a9-439a-93d4-6ba1de7cd3e2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:25.389721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:25.395749 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 217/427] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:25.397169 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0654740-fc47-4638-83b5-0b2357d29cff None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:25.457325 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:25.511724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe9c9b-a5a9-439a-93d4-6ba1de7cd3e2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:25.530442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0654740-fc47-4638-83b5-0b2357d29cff None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:25.555102 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d90e6e-aa39-4314-b85d-6550bedcb0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:25.556432 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d90e6e-aa39-4314-b85d-6550bedcb0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:25.556724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d90e6e-aa39-4314-b85d-6550bedcb0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:25.557005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d90e6e-aa39-4314-b85d-6550bedcb0a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:25.558019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d90e6e-aa39-4314-b85d-6550bedcb0a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:25.558324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d90e6e-aa39-4314-b85d-6550bedcb0a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:25.560387 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d90e6e-aa39-4314-b85d-6550bedcb0a7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:25.567393 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27fb82-da0c-4976-842b-477942e143e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:25.569017 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27fb82-da0c-4976-842b-477942e143e0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:25.569412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27fb82-da0c-4976-842b-477942e143e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:25.569791 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27fb82-da0c-4976-842b-477942e143e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:25.570560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 211/428] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:25.570896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27fb82-da0c-4976-842b-477942e143e0 None admin] RBAC: Adding query filter params (group.id=d5308803b7ea4a8499209c30aa8d030d, role.id=c077ebdf7f214d7585dfc1570eecf882, scope.project.id=4d31d318708e4e9caa66106ce370a235) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:25.571374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27fb82-da0c-4976-842b-477942e143e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:25.571483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d90e6e-aa39-4314-b85d-6550bedcb0a7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:25.573074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27fb82-da0c-4976-842b-477942e143e0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:25.579793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 218/429] 10.222.0.67 () {56 vars in 1376 bytes} [Fri Jun 24 15:00:25 2022] GET /identity/v3/role_assignments?group.id=d5308803b7ea4a8499209c30aa8d030d&scope.project.id=4d31d318708e4e9caa66106ce370a235&role.id=c077ebdf7f214d7585dfc1570eecf882 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:25.580695 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27fb82-da0c-4976-842b-477942e143e0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:25.702621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:25.782004 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d90e6e-aa39-4314-b85d-6550bedcb0a7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:25.792677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27fb82-da0c-4976-842b-477942e143e0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:25.815600 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:25.829623 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc28e6c-a59f-4e0a-8b35-b93220e6be07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:25.837154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc28e6c-a59f-4e0a-8b35-b93220e6be07 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:25.837154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc28e6c-a59f-4e0a-8b35-b93220e6be07 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:25.837154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc28e6c-a59f-4e0a-8b35-b93220e6be07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:25.837401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549e034-afcf-4c0a-94cf-d60821b5dcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:25.837401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549e034-afcf-4c0a-94cf-d60821b5dcea None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:25.837573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549e034-afcf-4c0a-94cf-d60821b5dcea None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:25.838104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549e034-afcf-4c0a-94cf-d60821b5dcea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:25.845606 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc28e6c-a59f-4e0a-8b35-b93220e6be07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-8dc28e6c-a59f-4e0a-8b35-b93220e6be07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:25.856214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:25.857237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:25.858059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:25.858059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:25.858059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:25.858059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:25.858059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:25.858059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:25.858059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:25.858059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:25.858059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:25.858059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:25.858059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 15:00:25.858059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:25.860732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 212/430] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:25.867722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc28e6c-a59f-4e0a-8b35-b93220e6be07 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:25.938200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549e034-afcf-4c0a-94cf-d60821b5dcea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:25.940013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549e034-afcf-4c0a-94cf-d60821b5dcea None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:26.018329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc28e6c-a59f-4e0a-8b35-b93220e6be07 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:26.028345 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:26.062970 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208def8-f2c6-4e6b-a21b-fe02d8ddf834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:26.064280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208def8-f2c6-4e6b-a21b-fe02d8ddf834 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:26.064999 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208def8-f2c6-4e6b-a21b-fe02d8ddf834 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:26.065481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208def8-f2c6-4e6b-a21b-fe02d8ddf834 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:26.066393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208def8-f2c6-4e6b-a21b-fe02d8ddf834 None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:26.066854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208def8-f2c6-4e6b-a21b-fe02d8ddf834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:26.069985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208def8-f2c6-4e6b-a21b-fe02d8ddf834 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:26.078247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 219/431] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:26.081011 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 213/432] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:00:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:26.091232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549e034-afcf-4c0a-94cf-d60821b5dcea None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:26.091427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208def8-f2c6-4e6b-a21b-fe02d8ddf834 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:26.152700 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:26.218120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208def8-f2c6-4e6b-a21b-fe02d8ddf834 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:26.223109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549e034-afcf-4c0a-94cf-d60821b5dcea None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:26.260196 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80922bbd-60ac-458a-9d83-01d337845218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:26.261276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80922bbd-60ac-458a-9d83-01d337845218 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:26.261563 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80922bbd-60ac-458a-9d83-01d337845218 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:26.261840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80922bbd-60ac-458a-9d83-01d337845218 None admin] PATH_INFO: `/v3/projects/4d31d318708e4e9caa66106ce370a235` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:26.262706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b628a-8810-4a20-b5a2-b726bff3d6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:26.263046 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80922bbd-60ac-458a-9d83-01d337845218 None admin] RBAC: Authorizing `identity:get_project(project_id=4d31d318708e4e9caa66106ce370a235)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:26.263698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b628a-8810-4a20-b5a2-b726bff3d6eb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:26.264133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b628a-8810-4a20-b5a2-b726bff3d6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:26.264531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80922bbd-60ac-458a-9d83-01d337845218 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:26.264687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b628a-8810-4a20-b5a2-b726bff3d6eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:26.266544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 220/433] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:26 2022] GET /identity/v3/projects/4d31d318708e4e9caa66106ce370a235 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:26.271832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80922bbd-60ac-458a-9d83-01d337845218 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:26.303339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:26.345542 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b628a-8810-4a20-b5a2-b726bff3d6eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:26.346726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b628a-8810-4a20-b5a2-b726bff3d6eb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:26.394497 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:26.401072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80922bbd-60ac-458a-9d83-01d337845218 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:26.440875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ed073-481e-4b5b-ba56-c892a82dd596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:26.441738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ed073-481e-4b5b-ba56-c892a82dd596 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:26.442021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ed073-481e-4b5b-ba56-c892a82dd596 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:26.442297 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ed073-481e-4b5b-ba56-c892a82dd596 None admin] PATH_INFO: `/v3/groups/d5308803b7ea4a8499209c30aa8d030d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:26.443732 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ed073-481e-4b5b-ba56-c892a82dd596 None admin] RBAC: Authorizing `identity:get_group(group_id=d5308803b7ea4a8499209c30aa8d030d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:26.445235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ed073-481e-4b5b-ba56-c892a82dd596 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:26.446867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 221/434] 10.222.0.67 () {56 vars in 1180 bytes} [Fri Jun 24 15:00:26 2022] GET /identity/v3/groups/d5308803b7ea4a8499209c30aa8d030d => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:26.452768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0ed98-760d-4cee-8621-c61c3d65a303 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:26.453057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0ed98-760d-4cee-8621-c61c3d65a303 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:26.453465 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0ed98-760d-4cee-8621-c61c3d65a303 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:26.453572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 214/435] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 15:00:26.454182 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b628a-8810-4a20-b5a2-b726bff3d6eb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:26.458756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 222/436] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:26.458756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0ed98-760d-4cee-8621-c61c3d65a303 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:26.559620 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:26.564480 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0ed98-760d-4cee-8621-c61c3d65a303 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:26.566267 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b628a-8810-4a20-b5a2-b726bff3d6eb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:26.600954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618323b2-49b4-45d6-84e7-018dd3bba50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:26.601745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618323b2-49b4-45d6-84e7-018dd3bba50d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:26.602033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618323b2-49b4-45d6-84e7-018dd3bba50d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:26.602202 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3935fc3-8249-414a-bd52-158c8f912808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:26.602317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618323b2-49b4-45d6-84e7-018dd3bba50d None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:26.602854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3935fc3-8249-414a-bd52-158c8f912808 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:26.603136 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3935fc3-8249-414a-bd52-158c8f912808 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:26.603438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3935fc3-8249-414a-bd52-158c8f912808 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:26.603541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618323b2-49b4-45d6-84e7-018dd3bba50d None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:26.604257 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3935fc3-8249-414a-bd52-158c8f912808 None admin] RBAC: Adding query filter params (group.id=d5308803b7ea4a8499209c30aa8d030d, role.id=b6dd71d4043d43e9823ec25176863347, scope.project.id=4d31d318708e4e9caa66106ce370a235) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:26.604566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3935fc3-8249-414a-bd52-158c8f912808 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:26.604953 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618323b2-49b4-45d6-84e7-018dd3bba50d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:26.605972 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3935fc3-8249-414a-bd52-158c8f912808 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:26.606875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 223/437] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:26 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:26.607807 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b345b0-db23-403c-811c-c1e19aa58341 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:26.608093 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b345b0-db23-403c-811c-c1e19aa58341 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:26.608371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b345b0-db23-403c-811c-c1e19aa58341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:26.614013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 215/438] 10.222.0.67 () {56 vars in 1376 bytes} [Fri Jun 24 15:00:26 2022] GET /identity/v3/role_assignments?group.id=d5308803b7ea4a8499209c30aa8d030d&scope.project.id=4d31d318708e4e9caa66106ce370a235&role.id=b6dd71d4043d43e9823ec25176863347 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:26.615838 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3935fc3-8249-414a-bd52-158c8f912808 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:26.628584 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:26.635458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b3b345b0-db23-403c-811c-c1e19aa58341 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:26.652729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b3b345b0-db23-403c-811c-c1e19aa58341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:26.718787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 224/439] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:26.742593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b345b0-db23-403c-811c-c1e19aa58341 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:26.746542 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3935fc3-8249-414a-bd52-158c8f912808 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:26.786508 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316ba6f-b6b8-42fb-9caa-42803dd625ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:26.787704 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316ba6f-b6b8-42fb-9caa-42803dd625ca None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:26.787976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316ba6f-b6b8-42fb-9caa-42803dd625ca None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:26.788247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316ba6f-b6b8-42fb-9caa-42803dd625ca None admin] PATH_INFO: `/v3/groups/d5308803b7ea4a8499209c30aa8d030d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:26.789571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316ba6f-b6b8-42fb-9caa-42803dd625ca None admin] RBAC: Authorizing `identity:get_group(group_id=d5308803b7ea4a8499209c30aa8d030d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:26.791111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316ba6f-b6b8-42fb-9caa-42803dd625ca None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:26.793104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 216/440] 10.222.0.67 () {56 vars in 1180 bytes} [Fri Jun 24 15:00:26 2022] GET /identity/v3/groups/d5308803b7ea4a8499209c30aa8d030d => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:26.798650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316ba6f-b6b8-42fb-9caa-42803dd625ca None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:26.850917 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:26.861506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b345b0-db23-403c-811c-c1e19aa58341 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:26.905547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:26.913012 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fd9e9-2d67-4bf9-b2da-ab3dc8d7312a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:26.913829 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fd9e9-2d67-4bf9-b2da-ab3dc8d7312a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:26.919006 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fd9e9-2d67-4bf9-b2da-ab3dc8d7312a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:26.919006 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fd9e9-2d67-4bf9-b2da-ab3dc8d7312a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:26.928999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927fd9e9-2d67-4bf9-b2da-ab3dc8d7312a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:26.932910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316ba6f-b6b8-42fb-9caa-42803dd625ca None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-927fd9e9-2d67-4bf9-b2da-ab3dc8d7312a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:26.938194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:26.939104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:26.939949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:26.939949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:26.939949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:26.939949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:26.939949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:26.939949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:26.939949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:26.939949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:26.939949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:26.939949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:26.939949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:26.939949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:26.942149 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 225/441] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:00:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:26.949359 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fd9e9-2d67-4bf9-b2da-ab3dc8d7312a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:26.976141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271936a-65f3-45be-aa95-404ac69fda20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:26.977174 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271936a-65f3-45be-aa95-404ac69fda20 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:26.977928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271936a-65f3-45be-aa95-404ac69fda20 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:26.978561 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271936a-65f3-45be-aa95-404ac69fda20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:26.980119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271936a-65f3-45be-aa95-404ac69fda20 None admin] RBAC: Adding query filter params (group.id=d5308803b7ea4a8499209c30aa8d030d, role.id=b6dd71d4043d43e9823ec25176863347, scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:26.980905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271936a-65f3-45be-aa95-404ac69fda20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:26.983199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271936a-65f3-45be-aa95-404ac69fda20 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:26.992030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 217/442] 10.222.0.67 () {56 vars in 1376 bytes} [Fri Jun 24 15:00:26 2022] GET /identity/v3/role_assignments?group.id=d5308803b7ea4a8499209c30aa8d030d&scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487&role.id=b6dd71d4043d43e9823ec25176863347 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:27.074499 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fd9e9-2d67-4bf9-b2da-ab3dc8d7312a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:27.123168 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e98455-8ef2-48b9-ac92-ad709d4d1463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:27.123793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e98455-8ef2-48b9-ac92-ad709d4d1463 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:27.124085 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e98455-8ef2-48b9-ac92-ad709d4d1463 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:27.124431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e98455-8ef2-48b9-ac92-ad709d4d1463 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:27.125136 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e98455-8ef2-48b9-ac92-ad709d4d1463 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:27.125448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e98455-8ef2-48b9-ac92-ad709d4d1463 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:27.126879 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e98455-8ef2-48b9-ac92-ad709d4d1463 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:27.137680 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 226/443] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:27.144325 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271936a-65f3-45be-aa95-404ac69fda20 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:27.158215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7725b06-a904-489d-919c-4533846e7e3d None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:27.158813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7725b06-a904-489d-919c-4533846e7e3d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:27.159098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7725b06-a904-489d-919c-4533846e7e3d None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:27.161471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 227/444] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:00:27.166800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e7138-0518-484d-80fc-92e6211131c5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:27.167092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e7138-0518-484d-80fc-92e6211131c5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:27.167408 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e7138-0518-484d-80fc-92e6211131c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:27.187429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f58e7138-0518-484d-80fc-92e6211131c5 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:27.188640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f58e7138-0518-484d-80fc-92e6211131c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:27.207636 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:27.276921 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 228/445] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 15:00:27.286056 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271936a-65f3-45be-aa95-404ac69fda20 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:27.301080 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e7138-0518-484d-80fc-92e6211131c5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:27.324015 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78785139-d486-489d-a2ea-a1e26e9137b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:27.324672 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78785139-d486-489d-a2ea-a1e26e9137b0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:27.324955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78785139-d486-489d-a2ea-a1e26e9137b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:27.325244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78785139-d486-489d-a2ea-a1e26e9137b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:27.336667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:27.408651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78785139-d486-489d-a2ea-a1e26e9137b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:27.409981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78785139-d486-489d-a2ea-a1e26e9137b0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:27.422820 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e7138-0518-484d-80fc-92e6211131c5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:27.461623 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249d28d-e63f-46b3-ac5e-56744ad6b8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:27.462418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249d28d-e63f-46b3-ac5e-56744ad6b8bb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:27.462701 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249d28d-e63f-46b3-ac5e-56744ad6b8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:27.462983 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249d28d-e63f-46b3-ac5e-56744ad6b8bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:27.472846 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249d28d-e63f-46b3-ac5e-56744ad6b8bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-3249d28d-e63f-46b3-ac5e-56744ad6b8bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:27.480578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:27.481554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:27.482370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:27.482370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:27.482370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:27.482370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:27.482370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:27.482370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:27.482370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:27.482370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:27.482370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:27.482370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:27.482370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:00:27.482370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:27.482370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 229/446] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:00:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:27.487420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249d28d-e63f-46b3-ac5e-56744ad6b8bb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:27.498404 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:27.517510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 218/447] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:27.523153 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78785139-d486-489d-a2ea-a1e26e9137b0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:27.562890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:27.588571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249d28d-e63f-46b3-ac5e-56744ad6b8bb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:27.619990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78785139-d486-489d-a2ea-a1e26e9137b0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:27.622539 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d2aa7-706b-4b1f-958c-6866733a1489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:27.623223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d2aa7-706b-4b1f-958c-6866733a1489 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:27.623557 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d2aa7-706b-4b1f-958c-6866733a1489 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:27.623838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d2aa7-706b-4b1f-958c-6866733a1489 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:27.624579 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d2aa7-706b-4b1f-958c-6866733a1489 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:27.624886 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d2aa7-706b-4b1f-958c-6866733a1489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:27.626251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d2aa7-706b-4b1f-958c-6866733a1489 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:27.634920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 230/448] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:27.640354 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d2aa7-706b-4b1f-958c-6866733a1489 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:27.654527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5af91-dd64-426c-8520-b582fe92e151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:27.655433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5af91-dd64-426c-8520-b582fe92e151 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:27.655719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5af91-dd64-426c-8520-b582fe92e151 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:27.655997 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5af91-dd64-426c-8520-b582fe92e151 None admin] PATH_INFO: `/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:27.657172 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5af91-dd64-426c-8520-b582fe92e151 None admin] RBAC: Authorizing `identity:get_project(project_id=01d2a11c551c4d8fbf8eb173cd1cc487)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:27.658612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5af91-dd64-426c-8520-b582fe92e151 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:27.660369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 219/449] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:27 2022] GET /identity/v3/projects/01d2a11c551c4d8fbf8eb173cd1cc487 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:27.666104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5af91-dd64-426c-8520-b582fe92e151 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:27.707887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:27.739199 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d2aa7-706b-4b1f-958c-6866733a1489 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:27.765414 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5af91-dd64-426c-8520-b582fe92e151 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:27.774582 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811ad9e-87b4-4712-922a-57f3d46e292f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:27.775266 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811ad9e-87b4-4712-922a-57f3d46e292f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:27.775410 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:27.775554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811ad9e-87b4-4712-922a-57f3d46e292f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:27.775828 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811ad9e-87b4-4712-922a-57f3d46e292f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:27.799032 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ffad4-eaf1-4c49-92c6-8804e8e1c3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:27.799923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ffad4-eaf1-4c49-92c6-8804e8e1c3bb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:27.800207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ffad4-eaf1-4c49-92c6-8804e8e1c3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:27.800483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ffad4-eaf1-4c49-92c6-8804e8e1c3bb None admin] PATH_INFO: `/v3/groups/d5308803b7ea4a8499209c30aa8d030d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:27.801667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ffad4-eaf1-4c49-92c6-8804e8e1c3bb None admin] RBAC: Authorizing `identity:get_group(group_id=d5308803b7ea4a8499209c30aa8d030d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:27.803438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ffad4-eaf1-4c49-92c6-8804e8e1c3bb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:27.805050 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 220/450] 10.222.0.67 () {56 vars in 1180 bytes} [Fri Jun 24 15:00:27 2022] GET /identity/v3/groups/d5308803b7ea4a8499209c30aa8d030d => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:27.810966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ffad4-eaf1-4c49-92c6-8804e8e1c3bb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:27.845214 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811ad9e-87b4-4712-922a-57f3d46e292f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:27.846762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811ad9e-87b4-4712-922a-57f3d46e292f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:27.907106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ffad4-eaf1-4c49-92c6-8804e8e1c3bb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:27.914740 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:27.942245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00e21a-6b3c-4c46-98bf-fbcc4a6114b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:27.942959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00e21a-6b3c-4c46-98bf-fbcc4a6114b6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:27.943269 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00e21a-6b3c-4c46-98bf-fbcc4a6114b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:27.943548 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00e21a-6b3c-4c46-98bf-fbcc4a6114b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:27.944437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00e21a-6b3c-4c46-98bf-fbcc4a6114b6 None admin] RBAC: Adding query filter params (group.id=d5308803b7ea4a8499209c30aa8d030d, role.id=c077ebdf7f214d7585dfc1570eecf882, scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:27.944777 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00e21a-6b3c-4c46-98bf-fbcc4a6114b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:27.946205 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00e21a-6b3c-4c46-98bf-fbcc4a6114b6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:27.946756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 231/451] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 15:00:27.967223 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 221/452] 10.222.0.67 () {56 vars in 1376 bytes} [Fri Jun 24 15:00:27 2022] GET /identity/v3/role_assignments?group.id=d5308803b7ea4a8499209c30aa8d030d&scope.project.id=01d2a11c551c4d8fbf8eb173cd1cc487&role.id=c077ebdf7f214d7585dfc1570eecf882 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:27.967445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811ad9e-87b4-4712-922a-57f3d46e292f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:28.056183 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811ad9e-87b4-4712-922a-57f3d46e292f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:28.093262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda70ab-235c-4202-a67b-a3b3e6535a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:28.094143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda70ab-235c-4202-a67b-a3b3e6535a55 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:28.094419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda70ab-235c-4202-a67b-a3b3e6535a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:28.094690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda70ab-235c-4202-a67b-a3b3e6535a55 None admin] PATH_INFO: `/v3/projects/4e36bcd54fa5438c93e449d16bda5ca3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:28.095912 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda70ab-235c-4202-a67b-a3b3e6535a55 None admin] RBAC: Authorizing `identity:get_project(project_id=4e36bcd54fa5438c93e449d16bda5ca3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:28.097355 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda70ab-235c-4202-a67b-a3b3e6535a55 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:28.099870 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 232/453] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:00:27 2022] GET /identity/v3/projects/4e36bcd54fa5438c93e449d16bda5ca3 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:28.105607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00e21a-6b3c-4c46-98bf-fbcc4a6114b6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:28.153258 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:28.213629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00e21a-6b3c-4c46-98bf-fbcc4a6114b6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:28.247276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836698b-f87d-4ae0-ba62-2cb0deec9ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:28.248141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836698b-f87d-4ae0-ba62-2cb0deec9ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:28.248426 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836698b-f87d-4ae0-ba62-2cb0deec9ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:28.248711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836698b-f87d-4ae0-ba62-2cb0deec9ae6 None admin] PATH_INFO: `/v3/groups/9e059c031abd45e2a952d8771961b440` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:28.249886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836698b-f87d-4ae0-ba62-2cb0deec9ae6 None admin] RBAC: Authorizing `identity:get_group(group_id=9e059c031abd45e2a952d8771961b440)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:28.251345 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836698b-f87d-4ae0-ba62-2cb0deec9ae6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:28.252968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 222/454] 10.222.0.67 () {56 vars in 1180 bytes} [Fri Jun 24 15:00:28 2022] GET /identity/v3/groups/9e059c031abd45e2a952d8771961b440 => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:28.258858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda70ab-235c-4202-a67b-a3b3e6535a55 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:28.300893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:28.358999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda70ab-235c-4202-a67b-a3b3e6535a55 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:28.391033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d650b11-fa46-41d6-8432-83408e26a21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:28.391706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650b11-fa46-41d6-8432-83408e26a21f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:28.391991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650b11-fa46-41d6-8432-83408e26a21f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:28.392271 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650b11-fa46-41d6-8432-83408e26a21f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:28.393126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d650b11-fa46-41d6-8432-83408e26a21f None admin] RBAC: Adding query filter params (group.id=9e059c031abd45e2a952d8771961b440, role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=4e36bcd54fa5438c93e449d16bda5ca3) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:28.393439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d650b11-fa46-41d6-8432-83408e26a21f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:28.394858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d650b11-fa46-41d6-8432-83408e26a21f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:28.401120 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 233/455] 10.222.0.67 () {56 vars in 1376 bytes} [Fri Jun 24 15:00:28 2022] GET /identity/v3/role_assignments?group.id=9e059c031abd45e2a952d8771961b440&scope.project.id=4e36bcd54fa5438c93e449d16bda5ca3&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:30.854874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a235d63-b25f-489c-afb5-6db0615484cb None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:30.855416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a235d63-b25f-489c-afb5-6db0615484cb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:30.855524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a235d63-b25f-489c-afb5-6db0615484cb None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:30.856829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 223/456] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:30.862376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8529c-895e-4644-afbf-11c07b4768ca None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:30.862804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8529c-895e-4644-afbf-11c07b4768ca None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:30.862980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8529c-895e-4644-afbf-11c07b4768ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:30.884218 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-02a8529c-895e-4644-afbf-11c07b4768ca None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:30.885426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-02a8529c-895e-4644-afbf-11c07b4768ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:30.980537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 234/457] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:31.017736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a235d63-b25f-489c-afb5-6db0615484cb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:31.112937 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1e47b-1ec0-42be-a80d-c9aef970fa36 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:31.113513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1e47b-1ec0-42be-a80d-c9aef970fa36 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:31.114004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1e47b-1ec0-42be-a80d-c9aef970fa36 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:31.116451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 235/458] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:31.134672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bc1f7-390a-45c6-9466-4d7cc0acca1a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:31.150662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bc1f7-390a-45c6-9466-4d7cc0acca1a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:31.150662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bc1f7-390a-45c6-9466-4d7cc0acca1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:31.166090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9f0bc1f7-390a-45c6-9466-4d7cc0acca1a None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:31.167619 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9f0bc1f7-390a-45c6-9466-4d7cc0acca1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:31.167864 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a235d63-b25f-489c-afb5-6db0615484cb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:31.182331 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:31.201945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:31.232230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666d21f-6d87-4b92-823e-00d5b5007dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:31.233283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666d21f-6d87-4b92-823e-00d5b5007dcd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:31.233715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666d21f-6d87-4b92-823e-00d5b5007dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:31.234157 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666d21f-6d87-4b92-823e-00d5b5007dcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:31.251843 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666d21f-6d87-4b92-823e-00d5b5007dcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:31.254848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666d21f-6d87-4b92-823e-00d5b5007dcd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:31.285151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a666d21f-6d87-4b92-823e-00d5b5007dcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:31.285151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:31.285151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:31.285151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:31.285151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:31.285151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:31.285151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:31.285151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:31.285151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:31.285151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:31.285151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:31.285151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:31.285151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:31.285151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:31.285151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:31.286826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:31.287748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:31.287748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:31.287748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:31.287748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:31.289571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 224/459] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:31.295323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 236/460] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 15:00:31.295471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666d21f-6d87-4b92-823e-00d5b5007dcd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:31.324999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bc1f7-390a-45c6-9466-4d7cc0acca1a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:31.428525 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666d21f-6d87-4b92-823e-00d5b5007dcd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:31.468364 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b76112-b1ed-465a-8cd2-15f506ed09c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:31.469231 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b76112-b1ed-465a-8cd2-15f506ed09c8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:31.469610 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b76112-b1ed-465a-8cd2-15f506ed09c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:31.469981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b76112-b1ed-465a-8cd2-15f506ed09c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:31.470819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b76112-b1ed-465a-8cd2-15f506ed09c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:31.471282 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b76112-b1ed-465a-8cd2-15f506ed09c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:31.472784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b76112-b1ed-465a-8cd2-15f506ed09c8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:31.486493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 225/461] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:31.487862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a90cbe-5c2a-42c8-877d-b77da555a3e0 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:31.488145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a90cbe-5c2a-42c8-877d-b77da555a3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:31.488415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a90cbe-5c2a-42c8-877d-b77da555a3e0 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:31.493123 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 226/462] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:31.494379 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446af14-1ea7-4e79-ab1f-3399a3d83799 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:31.494661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446af14-1ea7-4e79-ab1f-3399a3d83799 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:31.494933 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446af14-1ea7-4e79-ab1f-3399a3d83799 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:31.496252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 227/463] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:31.496925 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446af14-1ea7-4e79-ab1f-3399a3d83799 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:31.510827 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bc1f7-390a-45c6-9466-4d7cc0acca1a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:31.518956 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:31.554766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12a4b9-8a7f-492e-9093-42b0390e9048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:31.555647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12a4b9-8a7f-492e-9093-42b0390e9048 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:31.555934 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12a4b9-8a7f-492e-9093-42b0390e9048 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:31.556239 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12a4b9-8a7f-492e-9093-42b0390e9048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:31.567094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12a4b9-8a7f-492e-9093-42b0390e9048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:31.569201 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12a4b9-8a7f-492e-9093-42b0390e9048 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:31.586993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4e12a4b9-8a7f-492e-9093-42b0390e9048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:31.586993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:31.586993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:31.586993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:31.586993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:31.586993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:31.586993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:31.586993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:31.586993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:31.586993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:31.586993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:31.586993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:31.586993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:31.586993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:31.586993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:31.588057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:31.589126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:31.589126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:31.589126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:31.589126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:31.592878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 237/464] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:31.594069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5155330-860e-4968-aa90-37909ed1f830 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:31.594349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5155330-860e-4968-aa90-37909ed1f830 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:31.594633 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5155330-860e-4968-aa90-37909ed1f830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:31.608530 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446af14-1ea7-4e79-ab1f-3399a3d83799 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:31.610247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c5155330-860e-4968-aa90-37909ed1f830 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:31.611452 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c5155330-860e-4968-aa90-37909ed1f830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:31.620651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:31.647859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c312ad3-fe70-4c40-a37f-7554ae534d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:31.648529 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c312ad3-fe70-4c40-a37f-7554ae534d16 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:31.648814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c312ad3-fe70-4c40-a37f-7554ae534d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:31.649085 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c312ad3-fe70-4c40-a37f-7554ae534d16 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:31.649783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c312ad3-fe70-4c40-a37f-7554ae534d16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:31.651227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c312ad3-fe70-4c40-a37f-7554ae534d16 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:31.679827 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 238/465] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:31.681022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbc0c3-e09a-4fa9-89b8-7d52a814efe3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:31.681311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbc0c3-e09a-4fa9-89b8-7d52a814efe3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:31.681587 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbc0c3-e09a-4fa9-89b8-7d52a814efe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:31.697270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-77dbc0c3-e09a-4fa9-89b8-7d52a814efe3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:31.698441 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-77dbc0c3-e09a-4fa9-89b8-7d52a814efe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:31.724978 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 228/466] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 15:00:31 2022] POST /identity/v3/users => generated 246 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:00:31.725809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c312ad3-fe70-4c40-a37f-7554ae534d16 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:31.788698 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 239/467] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:31.796759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbc0c3-e09a-4fa9-89b8-7d52a814efe3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:31.816106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:31.839777 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c312ad3-fe70-4c40-a37f-7554ae534d16 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:31.880558 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346ee95-29f8-42bf-838f-c9c999224dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:31.881359 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346ee95-29f8-42bf-838f-c9c999224dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:31.881523 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346ee95-29f8-42bf-838f-c9c999224dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:31.881798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346ee95-29f8-42bf-838f-c9c999224dd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:31.882534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346ee95-29f8-42bf-838f-c9c999224dd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:31.883098 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346ee95-29f8-42bf-838f-c9c999224dd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:31.885707 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346ee95-29f8-42bf-838f-c9c999224dd3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:31.898659 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 229/468] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:31.899532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346ee95-29f8-42bf-838f-c9c999224dd3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:31.929184 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:31.961966 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbc0c3-e09a-4fa9-89b8-7d52a814efe3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:32.005987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97744ddf-4c90-42d9-a151-89e1ef0aff47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:32.006920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97744ddf-4c90-42d9-a151-89e1ef0aff47 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:32.007540 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97744ddf-4c90-42d9-a151-89e1ef0aff47 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:32.007825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97744ddf-4c90-42d9-a151-89e1ef0aff47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:32.025948 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97744ddf-4c90-42d9-a151-89e1ef0aff47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:32.028252 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97744ddf-4c90-42d9-a151-89e1ef0aff47 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:32.030536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346ee95-29f8-42bf-838f-c9c999224dd3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:32.053308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-97744ddf-4c90-42d9-a151-89e1ef0aff47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:32.053308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:32.053308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:32.053308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:32.053308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:32.053308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:32.053308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:32.053308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:32.053308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:32.053308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:32.053308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:32.053308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:32.053308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:32.053308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:32.053308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:32.054398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:32.055397 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:32.055397 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:32.055397 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:32.055397 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:32.056682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 240/469] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:32.057714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97744ddf-4c90-42d9-a151-89e1ef0aff47 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:32.073604 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8bbe7-cb3b-4ea7-ab84-068ecbfde238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:32.074734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8bbe7-cb3b-4ea7-ab84-068ecbfde238 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:32.075267 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8bbe7-cb3b-4ea7-ab84-068ecbfde238 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:32.075748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8bbe7-cb3b-4ea7-ab84-068ecbfde238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:32.079147 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:32.098524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8bbe7-cb3b-4ea7-ab84-068ecbfde238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:32.098524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8bbe7-cb3b-4ea7-ab84-068ecbfde238 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:32.116865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-25b8bbe7-cb3b-4ea7-ab84-068ecbfde238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:32.116865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:32.116865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:32.116865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:32.116865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:32.116865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:32.116865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:32.116865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:32.116865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:32.116865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:32.116865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:32.116865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:32.116865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:32.116865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:32.116865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:32.117756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:32.118634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:32.118634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:32.118634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:32.118634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:32.120710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 230/470] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:32.126205 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8bbe7-cb3b-4ea7-ab84-068ecbfde238 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:32.192489 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97744ddf-4c90-42d9-a151-89e1ef0aff47 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:32.218629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:32.228713 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc456b2-9a5e-482f-a91b-bbeb3b43f047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:32.229707 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc456b2-9a5e-482f-a91b-bbeb3b43f047 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:32.230128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc456b2-9a5e-482f-a91b-bbeb3b43f047 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:32.230592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc456b2-9a5e-482f-a91b-bbeb3b43f047 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:32.231639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc456b2-9a5e-482f-a91b-bbeb3b43f047 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:32.233243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc456b2-9a5e-482f-a91b-bbeb3b43f047 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:32.249152 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8bbe7-cb3b-4ea7-ab84-068ecbfde238 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:32.284201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e79e7-0051-4376-8159-4913b8def193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:32.284870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e79e7-0051-4376-8159-4913b8def193 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:32.285208 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e79e7-0051-4376-8159-4913b8def193 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:32.285485 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e79e7-0051-4376-8159-4913b8def193 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:32.286774 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e79e7-0051-4376-8159-4913b8def193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:32.287086 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e79e7-0051-4376-8159-4913b8def193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:32.288562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e79e7-0051-4376-8159-4913b8def193 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:32.304030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 231/471] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:32.304945 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e79e7-0051-4376-8159-4913b8def193 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:32.313115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 241/472] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 15:00:32 2022] POST /identity/v3/users => generated 248 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:00:32.315655 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc456b2-9a5e-482f-a91b-bbeb3b43f047 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:32.401772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:32.503916 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc456b2-9a5e-482f-a91b-bbeb3b43f047 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:32.514003 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e79e7-0051-4376-8159-4913b8def193 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:32.551446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0819a-3c20-49b2-88e1-a9e187d3a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:32.553180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0819a-3c20-49b2-88e1-a9e187d3a3b8 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:32.553609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0819a-3c20-49b2-88e1-a9e187d3a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:32.554041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0819a-3c20-49b2-88e1-a9e187d3a3b8 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:32.554874 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0819a-3c20-49b2-88e1-a9e187d3a3b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:32.556499 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0819a-3c20-49b2-88e1-a9e187d3a3b8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:32.562448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f93b2-3188-4477-be4d-ff486aab5514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:32.563819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f93b2-3188-4477-be4d-ff486aab5514 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:32.563819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f93b2-3188-4477-be4d-ff486aab5514 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:32.563990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f93b2-3188-4477-be4d-ff486aab5514 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:32.565435 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f93b2-3188-4477-be4d-ff486aab5514 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:32.565756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f93b2-3188-4477-be4d-ff486aab5514 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:32.567124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f93b2-3188-4477-be4d-ff486aab5514 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:32.585642 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 232/473] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:32.587437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa56bc-fa90-4437-9572-30205a4e42c7 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:32.587933 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa56bc-fa90-4437-9572-30205a4e42c7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:32.588210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa56bc-fa90-4437-9572-30205a4e42c7 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:32.589546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 233/474] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:32.590265 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa56bc-fa90-4437-9572-30205a4e42c7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:32.638724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 242/475] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 15:00:32 2022] POST /identity/v3/users => generated 248 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:00:32.640446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54655f05-ad8a-4c4b-b989-a2b1d2553e63 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:32.640742 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54655f05-ad8a-4c4b-b989-a2b1d2553e63 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:32.641017 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54655f05-ad8a-4c4b-b989-a2b1d2553e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:32.667683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-54655f05-ad8a-4c4b-b989-a2b1d2553e63 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:32.670198 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-54655f05-ad8a-4c4b-b989-a2b1d2553e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:32.714750 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:32.720715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa56bc-fa90-4437-9572-30205a4e42c7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:32.732634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:32.764893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253805a3-4fa3-4c35-9a0a-376c5215d81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:32.765639 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253805a3-4fa3-4c35-9a0a-376c5215d81a None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:32.765922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253805a3-4fa3-4c35-9a0a-376c5215d81a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:32.766268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253805a3-4fa3-4c35-9a0a-376c5215d81a None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:32.767067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253805a3-4fa3-4c35-9a0a-376c5215d81a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:32.771479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253805a3-4fa3-4c35-9a0a-376c5215d81a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:32.776540 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 243/476] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:32.800269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54655f05-ad8a-4c4b-b989-a2b1d2553e63 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:32.899089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 234/477] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 15:00:32 2022] POST /identity/v3/users => generated 249 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:00:32.916352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54655f05-ad8a-4c4b-b989-a2b1d2553e63 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:32.959241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473e449-7327-469c-ac98-e0b95e17ba4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:32.960110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473e449-7327-469c-ac98-e0b95e17ba4a None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:32.960404 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473e449-7327-469c-ac98-e0b95e17ba4a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:32.960683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473e449-7327-469c-ac98-e0b95e17ba4a None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:32.961467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473e449-7327-469c-ac98-e0b95e17ba4a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:32.963075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473e449-7327-469c-ac98-e0b95e17ba4a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:33.004939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.sql.core [None req-8473e449-7327-469c-ac98-e0b95e17ba4a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 15:00:33.004939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 15:00:33.004939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [parameters: {'id': '0f041994d2824973a6fc1ac185a7653e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 15:00:33.004939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80523) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 15:00:33.022256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8473e449-7327-469c-ac98-e0b95e17ba4a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 15:00:33.022256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:33.022256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 15:00:33.022256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 15:00:33.022256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 15:00:33.022256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 15:00:33.022256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 15:00:33.022256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 15:00:33.022256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 15:00:33.022256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application conn.query(q) Jun 24 15:00:33.022256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 15:00:33.022256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 15:00:33.022256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 15:00:33.022256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result.read() Jun 24 15:00:33.022256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return ref.to_dict() Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 15:00:33.023232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application next(self.gen) Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application yield resource Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application next(self.gen) Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise self.value Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application session.commit() Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 15:00:33.025013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.session.flush() Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application compat.raise_( Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 15:00:33.027412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 15:00:33.035946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application conn.query(q) Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result.read() Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 15:00:33.038273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [parameters: {'id': '0f041994d2824973a6fc1ac185a7653e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:33.039374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 15:00:33.040204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:33.041287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 244/478] 10.222.0.67 () {60 vars in 1131 bytes} [Fri Jun 24 15:00:32 2022] POST /identity/v3/roles => generated 144 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:33.041369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253805a3-4fa3-4c35-9a0a-376c5215d81a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:33.174101 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253805a3-4fa3-4c35-9a0a-376c5215d81a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:33.209513 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a2f31-b6f8-4ed4-9485-66d54d0061dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:33.210341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a2f31-b6f8-4ed4-9485-66d54d0061dc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:33.210625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a2f31-b6f8-4ed4-9485-66d54d0061dc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:33.210902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a2f31-b6f8-4ed4-9485-66d54d0061dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:33.221796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a2f31-b6f8-4ed4-9485-66d54d0061dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-fc1a2f31-b6f8-4ed4-9485-66d54d0061dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:33.230775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:33.231738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:33.232557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:33.232557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:33.232557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:33.232557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:33.232557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:33.232557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:33.232557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:33.232557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:33.232557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:33.232557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:33.232557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 15:00:33.232557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:33.232557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 235/479] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:00:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:33.237735 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473e449-7327-469c-ac98-e0b95e17ba4a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:33.273759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:33.345460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473e449-7327-469c-ac98-e0b95e17ba4a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:33.381768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4bc0e-4286-40e9-975b-39c44e1703d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:33.382442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4bc0e-4286-40e9-975b-39c44e1703d8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:33.382727 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4bc0e-4286-40e9-975b-39c44e1703d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:33.383004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4bc0e-4286-40e9-975b-39c44e1703d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:33.383793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4bc0e-4286-40e9-975b-39c44e1703d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:33.384101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4bc0e-4286-40e9-975b-39c44e1703d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:33.385608 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4bc0e-4286-40e9-975b-39c44e1703d8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:33.394007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 245/480] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:34.260540 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d656c5f-e477-4391-b366-57e6e593c10a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:34.261044 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d656c5f-e477-4391-b366-57e6e593c10a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:34.261152 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d656c5f-e477-4391-b366-57e6e593c10a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:34.262461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 236/481] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:34.267815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4816f6f-40da-45f3-a308-7b7e6a1c98dd None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:34.268213 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4816f6f-40da-45f3-a308-7b7e6a1c98dd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:34.268379 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4816f6f-40da-45f3-a308-7b7e6a1c98dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:34.292165 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b4816f6f-40da-45f3-a308-7b7e6a1c98dd None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:34.293359 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b4816f6f-40da-45f3-a308-7b7e6a1c98dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:34.366185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 246/482] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:34.401349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d656c5f-e477-4391-b366-57e6e593c10a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:34.436764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:34.537127 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d656c5f-e477-4391-b366-57e6e593c10a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:34.587346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38d692-0dc8-48f9-a3dc-b543ac2d42a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:34.588165 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38d692-0dc8-48f9-a3dc-b543ac2d42a7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:34.588451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38d692-0dc8-48f9-a3dc-b543ac2d42a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:34.588733 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38d692-0dc8-48f9-a3dc-b543ac2d42a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:34.611061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38d692-0dc8-48f9-a3dc-b543ac2d42a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-df38d692-0dc8-48f9-a3dc-b543ac2d42a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:34.619346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:34.621408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:34.623358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:34.623358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:34.623358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:34.623358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:34.623358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:34.623358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:34.623358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:34.623358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:34.623358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:34.623358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:34.623358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:34.623358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:34.627223 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 237/483] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:34.631167 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4816f6f-40da-45f3-a308-7b7e6a1c98dd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:34.662581 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:34.715808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6454f99-19dc-4565-a603-734727b8ceeb None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:34.716151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6454f99-19dc-4565-a603-734727b8ceeb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:34.716387 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6454f99-19dc-4565-a603-734727b8ceeb None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:34.718108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 238/484] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:00:34.723118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c47a2-f510-425e-9eae-bceec52db39f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:34.723429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c47a2-f510-425e-9eae-bceec52db39f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:34.724235 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c47a2-f510-425e-9eae-bceec52db39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:34.744180 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-239c47a2-f510-425e-9eae-bceec52db39f None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:34.745373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-239c47a2-f510-425e-9eae-bceec52db39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:34.809128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4816f6f-40da-45f3-a308-7b7e6a1c98dd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:34.824732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 239/485] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:34.849193 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c47a2-f510-425e-9eae-bceec52db39f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:34.871648 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:34.884682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaa6ca-08df-4bd3-bf4a-edb2005d8ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:34.885356 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaa6ca-08df-4bd3-bf4a-edb2005d8ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:34.885638 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaa6ca-08df-4bd3-bf4a-edb2005d8ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:34.887328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaa6ca-08df-4bd3-bf4a-edb2005d8ee5 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:34.887328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beaa6ca-08df-4bd3-bf4a-edb2005d8ee5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:34.887328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beaa6ca-08df-4bd3-bf4a-edb2005d8ee5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:34.888311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beaa6ca-08df-4bd3-bf4a-edb2005d8ee5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:34.904355 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 247/486] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:34.910059 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaa6ca-08df-4bd3-bf4a-edb2005d8ee5 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:35.023973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c47a2-f510-425e-9eae-bceec52db39f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:35.063544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:35.074315 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f769632-e4e3-4e02-840d-e85231361868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:35.075116 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f769632-e4e3-4e02-840d-e85231361868 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:35.075422 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f769632-e4e3-4e02-840d-e85231361868 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:35.075702 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f769632-e4e3-4e02-840d-e85231361868 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:35.079280 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaa6ca-08df-4bd3-bf4a-edb2005d8ee5 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:35.089788 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f769632-e4e3-4e02-840d-e85231361868 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-6f769632-e4e3-4e02-840d-e85231361868 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:35.100673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:35.101611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:35.102430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:35.102430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:35.102430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:35.102430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:35.102430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:35.102430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:35.102430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:35.102430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:35.102430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:35.102430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:35.102430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:35.102430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:35.108722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 240/487] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:35.108722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a8a13-d170-4a7a-9631-64aa82a4d9f6 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:35.108722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a8a13-d170-4a7a-9631-64aa82a4d9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:35.108722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a8a13-d170-4a7a-9631-64aa82a4d9f6 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:35.108722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 241/488] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:35.115929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a8a13-d170-4a7a-9631-64aa82a4d9f6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:35.137670 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a8f43b-8f3e-4ef8-8f51-021b17f336b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:35.138474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8f43b-8f3e-4ef8-8f51-021b17f336b5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:35.138760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8f43b-8f3e-4ef8-8f51-021b17f336b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:35.139040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8f43b-8f3e-4ef8-8f51-021b17f336b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:35.154976 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a8f43b-8f3e-4ef8-8f51-021b17f336b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:35.157452 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a8f43b-8f3e-4ef8-8f51-021b17f336b5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:35.182352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-26a8f43b-8f3e-4ef8-8f51-021b17f336b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:35.182352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:35.182352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:35.182352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:35.182352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:35.182352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:35.182352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:35.182352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:35.182352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:35.182352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:35.182352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:35.182352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:35.182352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:35.182352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:35.182352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:35.183471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:35.184335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:35.184335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:35.184335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:35.184335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:35.192083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 248/489] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:35.192866 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58078ebf-9534-4f83-89d6-3ebc1d5e7ad0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:35.193206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58078ebf-9534-4f83-89d6-3ebc1d5e7ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:35.193432 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58078ebf-9534-4f83-89d6-3ebc1d5e7ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:35.215383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-58078ebf-9534-4f83-89d6-3ebc1d5e7ad0 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:35.216870 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-58078ebf-9534-4f83-89d6-3ebc1d5e7ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:35.218084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:35.251420 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a8a13-d170-4a7a-9631-64aa82a4d9f6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:35.290838 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43adaa-4d03-4679-8f50-bec4f68c2066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:35.291613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43adaa-4d03-4679-8f50-bec4f68c2066 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:35.291881 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43adaa-4d03-4679-8f50-bec4f68c2066 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:35.292160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43adaa-4d03-4679-8f50-bec4f68c2066 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:35.292909 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43adaa-4d03-4679-8f50-bec4f68c2066 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:35.293216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43adaa-4d03-4679-8f50-bec4f68c2066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:35.294748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43adaa-4d03-4679-8f50-bec4f68c2066 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:35.308638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 242/490] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:35.308638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43adaa-4d03-4679-8f50-bec4f68c2066 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:35.317574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 249/491] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:35.319080 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58078ebf-9534-4f83-89d6-3ebc1d5e7ad0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:35.405258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:35.440489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43adaa-4d03-4679-8f50-bec4f68c2066 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:35.456451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58078ebf-9534-4f83-89d6-3ebc1d5e7ad0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:35.476563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72d2e3-abd9-4e75-b661-7c1cf90e347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:35.477289 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72d2e3-abd9-4e75-b661-7c1cf90e347f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:35.477579 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72d2e3-abd9-4e75-b661-7c1cf90e347f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:35.477856 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72d2e3-abd9-4e75-b661-7c1cf90e347f None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:35.478579 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72d2e3-abd9-4e75-b661-7c1cf90e347f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:35.478891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72d2e3-abd9-4e75-b661-7c1cf90e347f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:35.480538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72d2e3-abd9-4e75-b661-7c1cf90e347f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:35.483095 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:35.499205 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 243/492] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:35.499205 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72d2e3-abd9-4e75-b661-7c1cf90e347f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:35.507887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e5e9d-135a-4c74-a6cc-27a1e2b8d24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:35.508896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e5e9d-135a-4c74-a6cc-27a1e2b8d24f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:35.509630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e5e9d-135a-4c74-a6cc-27a1e2b8d24f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:35.510050 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e5e9d-135a-4c74-a6cc-27a1e2b8d24f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:35.525977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e5e9d-135a-4c74-a6cc-27a1e2b8d24f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:35.528477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e5e9d-135a-4c74-a6cc-27a1e2b8d24f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:35.549257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-b67e5e9d-135a-4c74-a6cc-27a1e2b8d24f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:35.549257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:35.549257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:35.549257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:35.549257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:35.549257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:35.549257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:35.549257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:35.549257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:35.549257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:35.549257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:35.549257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:35.549257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:35.549257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:35.549257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:35.550429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:35.551525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:35.551525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:35.551525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:35.551525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:35.557637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 250/493] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:35.557637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e5e9d-135a-4c74-a6cc-27a1e2b8d24f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:35.621979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72d2e3-abd9-4e75-b661-7c1cf90e347f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:35.662686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef352e-ab9f-40e2-8ce6-c7d642420c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:35.663753 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef352e-ab9f-40e2-8ce6-c7d642420c89 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:35.664173 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef352e-ab9f-40e2-8ce6-c7d642420c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:35.664603 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef352e-ab9f-40e2-8ce6-c7d642420c89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:35.678413 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e5e9d-135a-4c74-a6cc-27a1e2b8d24f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:35.680266 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef352e-ab9f-40e2-8ce6-c7d642420c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-fcef352e-ab9f-40e2-8ce6-c7d642420c89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:35.690754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:35.691898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:35.693737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:35.693737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:35.693737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:35.693737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:35.693737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:35.693737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:35.693737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:35.693737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:35.693737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:35.693737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:35.693737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:35.693737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:35.696764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 244/494] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:35.697703 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef352e-ab9f-40e2-8ce6-c7d642420c89 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:35.708077 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:35.723276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f47d5-d916-4f21-a6d0-818d0c9b76ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:35.724644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f47d5-d916-4f21-a6d0-818d0c9b76ab None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:35.725033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f47d5-d916-4f21-a6d0-818d0c9b76ab None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:35.725548 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f47d5-d916-4f21-a6d0-818d0c9b76ab None admin] PATH_INFO: `/v3/users/nova` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:35.736834 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f47d5-d916-4f21-a6d0-818d0c9b76ab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:35.739972 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f47d5-d916-4f21-a6d0-818d0c9b76ab None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-fb8f47d5-d916-4f21-a6d0-818d0c9b76ab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:35.757683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:35.758818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:35.759804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:35.759804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:35.759804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:35.759804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:35.759804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:35.759804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:35.759804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:35.759804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:35.759804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:00:35.759804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:35.759804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 251/495] 10.222.0.67 () {56 vars in 1093 bytes} [Fri Jun 24 15:00:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:35.759804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f47d5-d916-4f21-a6d0-818d0c9b76ab None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:35.771208 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:35.821470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef352e-ab9f-40e2-8ce6-c7d642420c89 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:35.856774 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cf70c-7274-48cf-82bc-fa4d06696ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:35.857438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cf70c-7274-48cf-82bc-fa4d06696ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:35.857722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cf70c-7274-48cf-82bc-fa4d06696ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:35.858003 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cf70c-7274-48cf-82bc-fa4d06696ae6 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:35.858717 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8cf70c-7274-48cf-82bc-fa4d06696ae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:35.859024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8cf70c-7274-48cf-82bc-fa4d06696ae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:35.860427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8cf70c-7274-48cf-82bc-fa4d06696ae6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:35.865065 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f47d5-d916-4f21-a6d0-818d0c9b76ab None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:35.877578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 245/496] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:35.877578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc390b-5bdf-49b3-aa45-aa8938d6415b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:35.877578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc390b-5bdf-49b3-aa45-aa8938d6415b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:35.877578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc390b-5bdf-49b3-aa45-aa8938d6415b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:35.877578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 246/497] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:35.877578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc390b-5bdf-49b3-aa45-aa8938d6415b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:35.906258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4029c64e-e903-474b-9d83-6e533e0264c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:35.906926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029c64e-e903-474b-9d83-6e533e0264c6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:35.907237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029c64e-e903-474b-9d83-6e533e0264c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:35.907519 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029c64e-e903-474b-9d83-6e533e0264c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:35.908264 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4029c64e-e903-474b-9d83-6e533e0264c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:35.908571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4029c64e-e903-474b-9d83-6e533e0264c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:35.909952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4029c64e-e903-474b-9d83-6e533e0264c6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:35.918945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 252/498] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:35.919544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4029c64e-e903-474b-9d83-6e533e0264c6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:35.962522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:35.988401 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc390b-5bdf-49b3-aa45-aa8938d6415b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:36.006433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:36.024478 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd48b3e-3219-45ae-bc91-5237a1b8c235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:36.025182 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd48b3e-3219-45ae-bc91-5237a1b8c235 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:36.025490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd48b3e-3219-45ae-bc91-5237a1b8c235 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:36.025793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd48b3e-3219-45ae-bc91-5237a1b8c235 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:36.026643 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd48b3e-3219-45ae-bc91-5237a1b8c235 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:36.027244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd48b3e-3219-45ae-bc91-5237a1b8c235 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:36.028678 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4029c64e-e903-474b-9d83-6e533e0264c6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:36.029270 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd48b3e-3219-45ae-bc91-5237a1b8c235 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:36.063758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bc573-7681-4ba7-99ed-550cfa02b24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:36.064558 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bc573-7681-4ba7-99ed-550cfa02b24c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:36.064843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bc573-7681-4ba7-99ed-550cfa02b24c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:36.065120 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bc573-7681-4ba7-99ed-550cfa02b24c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:36.071150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 247/499] 10.222.0.67 () {56 vars in 1133 bytes} [Fri Jun 24 15:00:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:36.071150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4521f6-a83a-4f3c-b1f3-22fa9625e04a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:36.071150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4521f6-a83a-4f3c-b1f3-22fa9625e04a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:36.071150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4521f6-a83a-4f3c-b1f3-22fa9625e04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:36.077769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bc573-7681-4ba7-99ed-550cfa02b24c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:36.079279 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bc573-7681-4ba7-99ed-550cfa02b24c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:36.085868 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3c4521f6-a83a-4f3c-b1f3-22fa9625e04a None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:36.087052 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3c4521f6-a83a-4f3c-b1f3-22fa9625e04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-2f6bc573-7681-4ba7-99ed-550cfa02b24c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:36.101490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:36.102427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:36.105474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:36.105474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:36.105474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:36.105474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:36.105474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:36.105474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:36.105474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:36.105474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:36.105474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 15:00:36.105474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:36.105474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 253/500] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:36.105474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bc573-7681-4ba7-99ed-550cfa02b24c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:36.163553 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 248/501] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:36.164729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4521f6-a83a-4f3c-b1f3-22fa9625e04a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:36.208875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bc573-7681-4ba7-99ed-550cfa02b24c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:36.210319 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:36.241799 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e0370-e391-4823-afc7-60b570bb4cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:36.242607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e0370-e391-4823-afc7-60b570bb4cfb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:36.242929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e0370-e391-4823-afc7-60b570bb4cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:36.243434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e0370-e391-4823-afc7-60b570bb4cfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:36.253802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e0370-e391-4823-afc7-60b570bb4cfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:36.256203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e0370-e391-4823-afc7-60b570bb4cfb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:36.265566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4521f6-a83a-4f3c-b1f3-22fa9625e04a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:36.267967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:36.273607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-be1e0370-e391-4823-afc7-60b570bb4cfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:36.273607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:36.273607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:36.273607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:36.273607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:36.273607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:36.273607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:36.273607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:36.273607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:36.273607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:36.273607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:36.273607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:36.273607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:36.273607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:36.273607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:36.274523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:36.279405 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:36.279405 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:36.279405 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:36.279405 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:36.279405 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 254/502] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:36.279405 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e0370-e391-4823-afc7-60b570bb4cfb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:36.299662 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a6dd4-e393-4eba-b172-09169e840dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:36.300311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a6dd4-e393-4eba-b172-09169e840dbb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:36.300590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a6dd4-e393-4eba-b172-09169e840dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:36.300864 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a6dd4-e393-4eba-b172-09169e840dbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:36.371427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a6dd4-e393-4eba-b172-09169e840dbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:36.372943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a6dd4-e393-4eba-b172-09169e840dbb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:36.377959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e0370-e391-4823-afc7-60b570bb4cfb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:36.411525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f96d34-54c2-4d44-b53e-26558f16a40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:36.412194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f96d34-54c2-4d44-b53e-26558f16a40f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:36.412476 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f96d34-54c2-4d44-b53e-26558f16a40f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:36.412749 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f96d34-54c2-4d44-b53e-26558f16a40f None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:36.413580 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f96d34-54c2-4d44-b53e-26558f16a40f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:36.413892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f96d34-54c2-4d44-b53e-26558f16a40f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:36.415329 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f96d34-54c2-4d44-b53e-26558f16a40f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:36.452429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 255/503] 10.222.0.67 () {56 vars in 1137 bytes} [Fri Jun 24 15:00:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:36.453525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79782e4-cb71-4c8b-a846-bd6f25862d92 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:36.453812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79782e4-cb71-4c8b-a846-bd6f25862d92 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:36.454089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79782e4-cb71-4c8b-a846-bd6f25862d92 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:36.455328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 256/504] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:36.456010 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79782e4-cb71-4c8b-a846-bd6f25862d92 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:36.470166 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:36.479693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 249/505] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:36.479693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a6dd4-e393-4eba-b172-09169e840dbb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:36.479693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:36.556079 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79782e4-cb71-4c8b-a846-bd6f25862d92 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:36.574453 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a6dd4-e393-4eba-b172-09169e840dbb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:36.589595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e6d24-7e8c-4592-afa9-42d5449fa647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:36.590387 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e6d24-7e8c-4592-afa9-42d5449fa647 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:36.590665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e6d24-7e8c-4592-afa9-42d5449fa647 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:36.590942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e6d24-7e8c-4592-afa9-42d5449fa647 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:36.601467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e6d24-7e8c-4592-afa9-42d5449fa647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:36.607485 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112eea53-d2be-4037-82f9-246c6e8656e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:36.608140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112eea53-d2be-4037-82f9-246c6e8656e5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:36.608424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112eea53-d2be-4037-82f9-246c6e8656e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:36.608701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112eea53-d2be-4037-82f9-246c6e8656e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-161e6d24-7e8c-4592-afa9-42d5449fa647 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:36.609127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:36.610037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:36.610852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:36.610852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:36.610852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:36.610852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:36.610852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:36.610852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:36.610852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:36.610852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:36.610852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:36.610852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:36.610852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:36.610852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:36.611650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112eea53-d2be-4037-82f9-246c6e8656e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:36.611650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112eea53-d2be-4037-82f9-246c6e8656e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:36.611770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 257/506] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:36.611770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e6d24-7e8c-4592-afa9-42d5449fa647 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:36.611896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112eea53-d2be-4037-82f9-246c6e8656e5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:36.626504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 250/507] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:36.628384 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fd3bd-fcd3-4422-9668-7a1c20db7e94 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:36.628678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fd3bd-fcd3-4422-9668-7a1c20db7e94 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:36.628952 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fd3bd-fcd3-4422-9668-7a1c20db7e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:36.646033 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f39fd3bd-fcd3-4422-9668-7a1c20db7e94 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:36.647386 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f39fd3bd-fcd3-4422-9668-7a1c20db7e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:36.682999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:36.704457 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:36.712955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e6d24-7e8c-4592-afa9-42d5449fa647 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:36.713079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 251/508] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:36.713746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fd3bd-fcd3-4422-9668-7a1c20db7e94 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:36.754476 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe3e9d-2866-43be-a2ed-da8323011876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:36.755798 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe3e9d-2866-43be-a2ed-da8323011876 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:36.756189 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe3e9d-2866-43be-a2ed-da8323011876 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:36.756477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe3e9d-2866-43be-a2ed-da8323011876 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:36.813713 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fd3bd-fcd3-4422-9668-7a1c20db7e94 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:36.823235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe3e9d-2866-43be-a2ed-da8323011876 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:36.824804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe3e9d-2866-43be-a2ed-da8323011876 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:36.847164 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0bf559-8f4d-474a-b55b-424faf76cef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:36.847987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bf559-8f4d-474a-b55b-424faf76cef6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:36.848266 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bf559-8f4d-474a-b55b-424faf76cef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:36.848674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bf559-8f4d-474a-b55b-424faf76cef6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:36.858650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0bf559-8f4d-474a-b55b-424faf76cef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:36.860119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0bf559-8f4d-474a-b55b-424faf76cef6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:36.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-5b0bf559-8f4d-474a-b55b-424faf76cef6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:36.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:36.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:36.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:36.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:36.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:36.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:36.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:36.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:36.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:36.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:36.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:36.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:36.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:36.877352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:36.878245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:36.879088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:36.879088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:36.879088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:36.879088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:36.879088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 252/509] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:36.879394 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0bf559-8f4d-474a-b55b-424faf76cef6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:36.896242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:36.925960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:36.926313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 258/510] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:36.927452 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe3e9d-2866-43be-a2ed-da8323011876 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:36.980280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0bf559-8f4d-474a-b55b-424faf76cef6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:37.014160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d2fbf-4c60-4111-a489-cbd0a8c5d1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:37.014756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d2fbf-4c60-4111-a489-cbd0a8c5d1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:37.015039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d2fbf-4c60-4111-a489-cbd0a8c5d1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:37.015341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d2fbf-4c60-4111-a489-cbd0a8c5d1f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:37.016084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d2fbf-4c60-4111-a489-cbd0a8c5d1f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:37.016411 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d2fbf-4c60-4111-a489-cbd0a8c5d1f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:37.017763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d2fbf-4c60-4111-a489-cbd0a8c5d1f4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:37.025401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe3e9d-2866-43be-a2ed-da8323011876 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:37.025856 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 253/511] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:37.026629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d2fbf-4c60-4111-a489-cbd0a8c5d1f4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:37.059274 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016f7e6-885d-4c42-bc29-89ecb895d0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:37.060068 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016f7e6-885d-4c42-bc29-89ecb895d0ce None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:37.060356 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016f7e6-885d-4c42-bc29-89ecb895d0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:37.060634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016f7e6-885d-4c42-bc29-89ecb895d0ce None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:37.069068 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016f7e6-885d-4c42-bc29-89ecb895d0ce None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:37.070510 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016f7e6-885d-4c42-bc29-89ecb895d0ce None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-c016f7e6-885d-4c42-bc29-89ecb895d0ce None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:37.084686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:37.085768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:37.086705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:37.086705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:37.086705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:37.086705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:37.086705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:37.086705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:37.086705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:37.086705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:37.086705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 15:00:37.086705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:37.086705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 259/512] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:37.086705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016f7e6-885d-4c42-bc29-89ecb895d0ce None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:37.118049 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:37.136430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d2fbf-4c60-4111-a489-cbd0a8c5d1f4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:37.145263 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:37.168590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fb9fb-f588-49a0-9a44-d907c7653100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:37.169378 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fb9fb-f588-49a0-9a44-d907c7653100 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:37.169660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fb9fb-f588-49a0-9a44-d907c7653100 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:37.169935 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fb9fb-f588-49a0-9a44-d907c7653100 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:37.182070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fb9fb-f588-49a0-9a44-d907c7653100 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:37.183571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fb9fb-f588-49a0-9a44-d907c7653100 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:37.189593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016f7e6-885d-4c42-bc29-89ecb895d0ce None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:37.200905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-866fb9fb-f588-49a0-9a44-d907c7653100 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:37.200905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:37.200905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:37.200905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:37.200905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:37.200905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:37.200905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:37.200905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:37.200905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:37.200905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:37.200905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:37.200905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:37.200905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:37.200905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:37.200905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:37.201770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:37.203260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:37.203260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:37.203260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:37.203260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:37.203260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 254/513] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:37.203260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fb9fb-f588-49a0-9a44-d907c7653100 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:37.229035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98b78e-fda9-42d9-b2dd-51f98d24e2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:37.229700 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98b78e-fda9-42d9-b2dd-51f98d24e2fb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:37.229992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98b78e-fda9-42d9-b2dd-51f98d24e2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:37.230274 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98b78e-fda9-42d9-b2dd-51f98d24e2fb None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:37.231006 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98b78e-fda9-42d9-b2dd-51f98d24e2fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:37.231351 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98b78e-fda9-42d9-b2dd-51f98d24e2fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:37.232733 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98b78e-fda9-42d9-b2dd-51f98d24e2fb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:37.244025 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 260/514] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:37.244810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98b78e-fda9-42d9-b2dd-51f98d24e2fb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:37.305745 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fb9fb-f588-49a0-9a44-d907c7653100 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:37.336341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:37.338848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce53433-2695-4283-ba24-d4634cee9c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:37.339690 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce53433-2695-4283-ba24-d4634cee9c1b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:37.339979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce53433-2695-4283-ba24-d4634cee9c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:37.340260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce53433-2695-4283-ba24-d4634cee9c1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:37.344713 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98b78e-fda9-42d9-b2dd-51f98d24e2fb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:37.351480 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce53433-2695-4283-ba24-d4634cee9c1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:37.352918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce53433-2695-4283-ba24-d4634cee9c1b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:37.370070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-1ce53433-2695-4283-ba24-d4634cee9c1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:37.370070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:37.370070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:37.370070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:37.370070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:37.370070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:37.370070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:37.370070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:37.370070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:37.370070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:37.370070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:37.370070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:37.370070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:37.370070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:37.370070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:37.371071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:37.371976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:37.371976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:37.371976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:37.371976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:37.371976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 255/515] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:37.372240 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce53433-2695-4283-ba24-d4634cee9c1b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:37.384143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd15f70-e492-45e1-a4b6-2edc233da9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:37.384938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd15f70-e492-45e1-a4b6-2edc233da9dd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:37.385222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd15f70-e492-45e1-a4b6-2edc233da9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:37.385500 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd15f70-e492-45e1-a4b6-2edc233da9dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:37.387644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:37.396344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd15f70-e492-45e1-a4b6-2edc233da9dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:37.401419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd15f70-e492-45e1-a4b6-2edc233da9dd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:37.415374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9dd15f70-e492-45e1-a4b6-2edc233da9dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:37.415374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:37.415374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:37.415374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:37.415374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:37.415374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:37.415374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:37.415374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:37.415374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:37.415374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:37.415374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:37.415374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:37.415374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:37.415374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:37.415374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:37.416243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:37.417126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:37.417126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:37.417126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:37.417126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:37.417126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 261/516] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:37.417401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd15f70-e492-45e1-a4b6-2edc233da9dd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:37.478406 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce53433-2695-4283-ba24-d4634cee9c1b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:37.512248 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89946a05-bc73-437f-88a7-9b442de87d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:37.512912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89946a05-bc73-437f-88a7-9b442de87d98 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:37.513195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89946a05-bc73-437f-88a7-9b442de87d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:37.513470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89946a05-bc73-437f-88a7-9b442de87d98 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:37.514302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89946a05-bc73-437f-88a7-9b442de87d98 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:37.514609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89946a05-bc73-437f-88a7-9b442de87d98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:37.516063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89946a05-bc73-437f-88a7-9b442de87d98 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:37.520335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd15f70-e492-45e1-a4b6-2edc233da9dd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:37.533797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 256/517] 10.222.0.67 () {56 vars in 1137 bytes} [Fri Jun 24 15:00:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:37.534610 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89946a05-bc73-437f-88a7-9b442de87d98 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:37.559159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d452c5-3159-4d12-8fb3-e9419a3b0331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:37.559883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d452c5-3159-4d12-8fb3-e9419a3b0331 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:37.560169 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d452c5-3159-4d12-8fb3-e9419a3b0331 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:37.560451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d452c5-3159-4d12-8fb3-e9419a3b0331 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:37.561185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d452c5-3159-4d12-8fb3-e9419a3b0331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:37.561493 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d452c5-3159-4d12-8fb3-e9419a3b0331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:37.562854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d452c5-3159-4d12-8fb3-e9419a3b0331 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:37.579770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 262/518] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:37.579770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d452c5-3159-4d12-8fb3-e9419a3b0331 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:37.581203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:37.606703 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:37.639473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89946a05-bc73-437f-88a7-9b442de87d98 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:37.674330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be8fce-4490-4d42-8fe4-193432f033d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:37.675123 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be8fce-4490-4d42-8fe4-193432f033d6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:37.675437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be8fce-4490-4d42-8fe4-193432f033d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:37.675725 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be8fce-4490-4d42-8fe4-193432f033d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:37.679620 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d452c5-3159-4d12-8fb3-e9419a3b0331 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:37.683470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be8fce-4490-4d42-8fe4-193432f033d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:37.684922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be8fce-4490-4d42-8fe4-193432f033d6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:37.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b0be8fce-4490-4d42-8fe4-193432f033d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:37.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:37.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:37.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:37.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:37.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:37.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:37.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:37.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:37.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:37.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:37.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:37.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:37.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:37.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:37.700020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:37.700877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:37.700877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:37.700877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:37.700877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:37.700877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:37.700877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:37.700877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:37.700877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:37.700877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 257/519] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:37.701341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be8fce-4490-4d42-8fe4-193432f033d6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:37.716233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd289ab-5d7a-494a-8e2a-c1440fe4a619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:37.716894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd289ab-5d7a-494a-8e2a-c1440fe4a619 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:37.717183 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd289ab-5d7a-494a-8e2a-c1440fe4a619 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:37.717458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd289ab-5d7a-494a-8e2a-c1440fe4a619 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:37.718189 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd289ab-5d7a-494a-8e2a-c1440fe4a619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:37.718488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd289ab-5d7a-494a-8e2a-c1440fe4a619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:37.719892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd289ab-5d7a-494a-8e2a-c1440fe4a619 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:37.732977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 263/520] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:37.733915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd289ab-5d7a-494a-8e2a-c1440fe4a619 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:37.802269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:37.808884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be8fce-4490-4d42-8fe4-193432f033d6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:37.841370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd289ab-5d7a-494a-8e2a-c1440fe4a619 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:37.841553 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:37.850122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a89e83-6b57-4ec9-9d97-aac5da2ff032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:37.851106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a89e83-6b57-4ec9-9d97-aac5da2ff032 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:37.852185 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a89e83-6b57-4ec9-9d97-aac5da2ff032 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:37.852185 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a89e83-6b57-4ec9-9d97-aac5da2ff032 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:37.853303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a89e83-6b57-4ec9-9d97-aac5da2ff032 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:37.853837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a89e83-6b57-4ec9-9d97-aac5da2ff032 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:37.856270 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a89e83-6b57-4ec9-9d97-aac5da2ff032 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:37.870842 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 258/521] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:37.871699 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a89e83-6b57-4ec9-9d97-aac5da2ff032 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:37.880276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d999d-32d8-48fd-aee5-bb50a2499601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:37.880941 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d999d-32d8-48fd-aee5-bb50a2499601 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:37.881231 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d999d-32d8-48fd-aee5-bb50a2499601 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:37.881512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d999d-32d8-48fd-aee5-bb50a2499601 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:37.951289 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d999d-32d8-48fd-aee5-bb50a2499601 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:37.952880 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d999d-32d8-48fd-aee5-bb50a2499601 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:37.985842 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a89e83-6b57-4ec9-9d97-aac5da2ff032 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:38.016611 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:38.021420 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f2a2b-2290-4015-8fbf-a589ebd97720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:38.022129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f2a2b-2290-4015-8fbf-a589ebd97720 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:38.022512 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f2a2b-2290-4015-8fbf-a589ebd97720 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:38.022726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f2a2b-2290-4015-8fbf-a589ebd97720 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:38.023459 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f2a2b-2290-4015-8fbf-a589ebd97720 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:38.024912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f2a2b-2290-4015-8fbf-a589ebd97720 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:38.057155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 264/522] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:38.058408 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d999d-32d8-48fd-aee5-bb50a2499601 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:38.106804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 259/523] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 15:00:37 2022] POST /identity/v3/users => generated 247 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:00:38.107730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f2a2b-2290-4015-8fbf-a589ebd97720 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:38.180924 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:38.186800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d999d-32d8-48fd-aee5-bb50a2499601 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:38.232405 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52999188-2b1a-4a8d-8811-ec7619e5593e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:38.233095 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52999188-2b1a-4a8d-8811-ec7619e5593e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:38.233384 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52999188-2b1a-4a8d-8811-ec7619e5593e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:38.233664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52999188-2b1a-4a8d-8811-ec7619e5593e None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:38.234486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52999188-2b1a-4a8d-8811-ec7619e5593e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:38.234804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52999188-2b1a-4a8d-8811-ec7619e5593e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:38.236339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52999188-2b1a-4a8d-8811-ec7619e5593e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:38.252132 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f2a2b-2290-4015-8fbf-a589ebd97720 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:38.257950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 265/524] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:38.260157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52999188-2b1a-4a8d-8811-ec7619e5593e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:38.307251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4918f62-2983-4e73-ba6f-2022bf95474c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:38.308475 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4918f62-2983-4e73-ba6f-2022bf95474c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:38.308967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4918f62-2983-4e73-ba6f-2022bf95474c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:38.309504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4918f62-2983-4e73-ba6f-2022bf95474c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:38.309929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:38.322864 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4918f62-2983-4e73-ba6f-2022bf95474c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:38.324731 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4918f62-2983-4e73-ba6f-2022bf95474c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:38.342311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f4918f62-2983-4e73-ba6f-2022bf95474c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:38.342311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:38.342311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:38.342311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:38.342311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:38.342311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:38.342311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:38.342311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:38.342311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:38.342311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:38.342311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:38.342311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:38.342311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:38.342311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:38.342311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:38.343728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:38.344787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:38.344787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:38.344787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:38.344787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:38.344787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:38.344787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:38.344787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:38.344787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:38.344787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 260/525] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:38.344787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4918f62-2983-4e73-ba6f-2022bf95474c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:38.404258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52999188-2b1a-4a8d-8811-ec7619e5593e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:38.449599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86238599-2047-43c1-a206-5791301587e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:38.451258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86238599-2047-43c1-a206-5791301587e6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:38.451854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86238599-2047-43c1-a206-5791301587e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:38.452424 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86238599-2047-43c1-a206-5791301587e6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:38.457421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4918f62-2983-4e73-ba6f-2022bf95474c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:38.463015 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86238599-2047-43c1-a206-5791301587e6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:38.464731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86238599-2047-43c1-a206-5791301587e6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:38.466551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-86238599-2047-43c1-a206-5791301587e6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:38.481612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:38.483243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:38.484722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:38.484722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:38.484722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:38.484722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:38.484722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:38.484722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:38.484722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:38.484722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:38.484722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 15:00:38.484722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:38.484722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 266/526] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:38.484722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86238599-2047-43c1-a206-5791301587e6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:38.502913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d6b5b-133a-429c-a333-1936372f5ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:38.504280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d6b5b-133a-429c-a333-1936372f5ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:38.504603 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d6b5b-133a-429c-a333-1936372f5ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:38.505460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d6b5b-133a-429c-a333-1936372f5ea0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:38.519671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d6b5b-133a-429c-a333-1936372f5ea0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:38.521658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d6b5b-133a-429c-a333-1936372f5ea0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:38.540495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-8f1d6b5b-133a-429c-a333-1936372f5ea0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:38.540495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:38.540495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:38.540495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:38.540495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:38.540495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:38.540495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:38.540495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:38.540495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:38.540495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:38.540495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:38.540495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:38.540495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:38.540495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:38.540495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:38.541599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:38.542496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:38.542496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:38.542496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:38.542496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:38.542496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 261/527] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:38.542496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d6b5b-133a-429c-a333-1936372f5ea0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:38.588785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:38.613875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86238599-2047-43c1-a206-5791301587e6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:38.643200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febac8c-b0ee-4ea1-9678-3f7ad9074aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:38.644378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febac8c-b0ee-4ea1-9678-3f7ad9074aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:38.644726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febac8c-b0ee-4ea1-9678-3f7ad9074aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:38.645008 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febac8c-b0ee-4ea1-9678-3f7ad9074aa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:38.645907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febac8c-b0ee-4ea1-9678-3f7ad9074aa6 None admin] RBAC: Adding query filter params (role.id=716cc96fb28146b7b9ae6271569a206f, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=51de59caa3144a1cbf12b20b15e2134f) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:38.646317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febac8c-b0ee-4ea1-9678-3f7ad9074aa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:38.648011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febac8c-b0ee-4ea1-9678-3f7ad9074aa6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:38.650265 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d6b5b-133a-429c-a333-1936372f5ea0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:38.664625 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 267/528] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:38 2022] GET /identity/v3/role_assignments?user.id=51de59caa3144a1cbf12b20b15e2134f&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=716cc96fb28146b7b9ae6271569a206f => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:38.664625 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febac8c-b0ee-4ea1-9678-3f7ad9074aa6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:38.691984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4db8bd-1456-4b6b-acd7-cb4dd20b20f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:38.694138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4db8bd-1456-4b6b-acd7-cb4dd20b20f9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:38.694429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4db8bd-1456-4b6b-acd7-cb4dd20b20f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:38.694706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4db8bd-1456-4b6b-acd7-cb4dd20b20f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:38.695899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4db8bd-1456-4b6b-acd7-cb4dd20b20f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:38.696215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4db8bd-1456-4b6b-acd7-cb4dd20b20f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:38.698573 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4db8bd-1456-4b6b-acd7-cb4dd20b20f9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:38.723748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 262/529] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:38.724638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4db8bd-1456-4b6b-acd7-cb4dd20b20f9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:38.755312 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:38.805587 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febac8c-b0ee-4ea1-9678-3f7ad9074aa6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:38.848239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4db8bd-1456-4b6b-acd7-cb4dd20b20f9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:38.857903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f654b-cb60-4f08-9b91-153b17d9fdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:38.859262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f654b-cb60-4f08-9b91-153b17d9fdfe None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:38.860091 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f654b-cb60-4f08-9b91-153b17d9fdfe None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:38.860520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f654b-cb60-4f08-9b91-153b17d9fdfe None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:38.861775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f654b-cb60-4f08-9b91-153b17d9fdfe None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:38.862252 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f654b-cb60-4f08-9b91-153b17d9fdfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:38.864643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f654b-cb60-4f08-9b91-153b17d9fdfe None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:38.885591 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e405090-4df7-45b9-9a0a-a931be61efdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:38.886373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405090-4df7-45b9-9a0a-a931be61efdd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:38.886730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405090-4df7-45b9-9a0a-a931be61efdd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:38.887069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405090-4df7-45b9-9a0a-a931be61efdd None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:38.887943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e405090-4df7-45b9-9a0a-a931be61efdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:38.888298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e405090-4df7-45b9-9a0a-a931be61efdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:38.888460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 268/530] 10.222.0.67 () {56 vars in 1139 bytes} [Fri Jun 24 15:00:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:38.889417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f654b-cb60-4f08-9b91-153b17d9fdfe None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:38.889922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e405090-4df7-45b9-9a0a-a931be61efdd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:38.901234 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:38.906193 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 263/531] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:38.910527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e405090-4df7-45b9-9a0a-a931be61efdd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:38.995511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:39.018782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e405090-4df7-45b9-9a0a-a931be61efdd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:39.025524 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f654b-cb60-4f08-9b91-153b17d9fdfe None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:39.057483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6243f0-2ea1-4464-ac7c-9a07da48dc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:39.058151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6243f0-2ea1-4464-ac7c-9a07da48dc3b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:39.058836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6243f0-2ea1-4464-ac7c-9a07da48dc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:39.059127 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6243f0-2ea1-4464-ac7c-9a07da48dc3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:39.068658 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53299a-a6f8-498e-ab28-b0fc34eef48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:39.070108 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53299a-a6f8-498e-ab28-b0fc34eef48f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:39.070537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53299a-a6f8-498e-ab28-b0fc34eef48f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:39.070957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53299a-a6f8-498e-ab28-b0fc34eef48f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:39.072420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53299a-a6f8-498e-ab28-b0fc34eef48f None admin] RBAC: Adding query filter params (role.id=716cc96fb28146b7b9ae6271569a206f, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=1971c464293f426f97d6cb199e48193b) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:39.072881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53299a-a6f8-498e-ab28-b0fc34eef48f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:39.075269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53299a-a6f8-498e-ab28-b0fc34eef48f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:39.083581 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 269/532] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:38 2022] GET /identity/v3/role_assignments?user.id=1971c464293f426f97d6cb199e48193b&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=716cc96fb28146b7b9ae6271569a206f => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:39.085220 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53299a-a6f8-498e-ab28-b0fc34eef48f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:39.195374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6243f0-2ea1-4464-ac7c-9a07da48dc3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:39.197451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6243f0-2ea1-4464-ac7c-9a07da48dc3b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:39.227559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:39.255515 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53299a-a6f8-498e-ab28-b0fc34eef48f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:39.296187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531787d5-277e-4ef6-8168-ecc448acef15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:39.296991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531787d5-277e-4ef6-8168-ecc448acef15 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:39.297276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531787d5-277e-4ef6-8168-ecc448acef15 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:39.297554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531787d5-277e-4ef6-8168-ecc448acef15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:39.305245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531787d5-277e-4ef6-8168-ecc448acef15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:39.306837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531787d5-277e-4ef6-8168-ecc448acef15 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:39.320389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:39.323331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-531787d5-277e-4ef6-8168-ecc448acef15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:39.323331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:39.323331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:39.323331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:39.323331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:39.323331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:39.323331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:39.323331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:39.323331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:39.323331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:39.323331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:39.323331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:39.323331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:39.323331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:39.323331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:39.324157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:39.325011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:39.325011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:39.325011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:39.325011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:39.325011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:39.325011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:39.325011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:39.325011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:39.325011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 270/533] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:39.332261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531787d5-277e-4ef6-8168-ecc448acef15 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:39.374643 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 264/534] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:39.380109 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6243f0-2ea1-4464-ac7c-9a07da48dc3b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:39.515086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531787d5-277e-4ef6-8168-ecc448acef15 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:39.546083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:39.554460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc7256-e0a0-49bc-a16a-8ed06c325ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:39.555140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc7256-e0a0-49bc-a16a-8ed06c325ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:39.555458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc7256-e0a0-49bc-a16a-8ed06c325ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:39.557649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc7256-e0a0-49bc-a16a-8ed06c325ba4 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:39.557649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc7256-e0a0-49bc-a16a-8ed06c325ba4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:39.557649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc7256-e0a0-49bc-a16a-8ed06c325ba4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:39.558983 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc7256-e0a0-49bc-a16a-8ed06c325ba4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:39.562837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6243f0-2ea1-4464-ac7c-9a07da48dc3b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:39.581696 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 271/535] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:39.589160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc7256-e0a0-49bc-a16a-8ed06c325ba4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:39.628472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eda911-b42a-4ab4-beb2-11b4ba3bf22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:39.630965 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eda911-b42a-4ab4-beb2-11b4ba3bf22c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:39.633247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eda911-b42a-4ab4-beb2-11b4ba3bf22c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:39.633730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eda911-b42a-4ab4-beb2-11b4ba3bf22c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:39.655119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eda911-b42a-4ab4-beb2-11b4ba3bf22c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:39.656906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eda911-b42a-4ab4-beb2-11b4ba3bf22c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:39.684967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-65eda911-b42a-4ab4-beb2-11b4ba3bf22c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:39.684967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:39.684967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:39.684967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:39.684967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:39.684967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:39.684967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:39.684967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:39.684967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:39.684967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:39.684967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:39.684967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:39.684967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:39.684967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:39.684967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:39.685982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:39.686835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:39.686835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:39.686835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:39.686835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:39.686835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 265/536] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:39.696396 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eda911-b42a-4ab4-beb2-11b4ba3bf22c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:39.736225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc7256-e0a0-49bc-a16a-8ed06c325ba4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:39.751505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:39.801241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80ecd7-6091-4f41-9ec2-a673c3822a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:39.802425 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80ecd7-6091-4f41-9ec2-a673c3822a21 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:39.802964 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80ecd7-6091-4f41-9ec2-a673c3822a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:39.803267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80ecd7-6091-4f41-9ec2-a673c3822a21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:39.804099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80ecd7-6091-4f41-9ec2-a673c3822a21 None admin] RBAC: Adding query filter params (role.id=716cc96fb28146b7b9ae6271569a206f, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=2efb120c084a48858d32f97843d76834) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:39.804415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80ecd7-6091-4f41-9ec2-a673c3822a21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:39.805848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80ecd7-6091-4f41-9ec2-a673c3822a21 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:39.819486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 272/537] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:39 2022] GET /identity/v3/role_assignments?user.id=2efb120c084a48858d32f97843d76834&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=716cc96fb28146b7b9ae6271569a206f => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:39.851661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eda911-b42a-4ab4-beb2-11b4ba3bf22c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:39.889558 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cb4b3-782d-4748-823b-b0a244ac6840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:39.890239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cb4b3-782d-4748-823b-b0a244ac6840 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:39.890520 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cb4b3-782d-4748-823b-b0a244ac6840 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:39.890798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cb4b3-782d-4748-823b-b0a244ac6840 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:39.891570 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cb4b3-782d-4748-823b-b0a244ac6840 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:39.891875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cb4b3-782d-4748-823b-b0a244ac6840 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:39.893281 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cb4b3-782d-4748-823b-b0a244ac6840 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:39.905947 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 266/538] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:39.911997 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80ecd7-6091-4f41-9ec2-a673c3822a21 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:39.975484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:40.051946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80ecd7-6091-4f41-9ec2-a673c3822a21 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:40.121715 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd89f5-2f87-46c7-a85b-392a166e1e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:40.122610 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd89f5-2f87-46c7-a85b-392a166e1e85 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:40.122902 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd89f5-2f87-46c7-a85b-392a166e1e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:40.123203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd89f5-2f87-46c7-a85b-392a166e1e85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:40.130707 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd89f5-2f87-46c7-a85b-392a166e1e85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:40.132186 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd89f5-2f87-46c7-a85b-392a166e1e85 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:40.159803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-aecd89f5-2f87-46c7-a85b-392a166e1e85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:40.159803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:40.159803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:40.159803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:40.159803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:40.159803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:40.159803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:40.159803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:40.159803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:40.159803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:40.159803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:40.159803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:40.159803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:40.159803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:40.159803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:40.160720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:40.162155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:40.162155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:40.162155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:40.162155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:40.162155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:40.162155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:40.162155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:40.162155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:40.162155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 273/539] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:40.167674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cb4b3-782d-4748-823b-b0a244ac6840 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:40.281289 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:40.291409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cb4b3-782d-4748-823b-b0a244ac6840 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:40.329474 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc714cb-823c-4f88-a571-c565a2421333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:40.330094 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc714cb-823c-4f88-a571-c565a2421333 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:40.330337 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc714cb-823c-4f88-a571-c565a2421333 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:40.330614 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc714cb-823c-4f88-a571-c565a2421333 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:40.331432 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc714cb-823c-4f88-a571-c565a2421333 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:40.331747 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc714cb-823c-4f88-a571-c565a2421333 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:40.333804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc714cb-823c-4f88-a571-c565a2421333 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:40.345865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 267/540] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:40.352506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd89f5-2f87-46c7-a85b-392a166e1e85 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:40.472675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd89f5-2f87-46c7-a85b-392a166e1e85 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:40.494544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:40.521841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9b552-cde5-4439-abe6-ef916e696a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:40.522621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9b552-cde5-4439-abe6-ef916e696a96 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:40.522790 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9b552-cde5-4439-abe6-ef916e696a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:40.523073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9b552-cde5-4439-abe6-ef916e696a96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:40.523937 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9b552-cde5-4439-abe6-ef916e696a96 None admin] RBAC: Adding query filter params (role.id=716cc96fb28146b7b9ae6271569a206f, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=95e7fb8f48dd4348b54e3694a8410bfd) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:40.524256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9b552-cde5-4439-abe6-ef916e696a96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:40.525697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9b552-cde5-4439-abe6-ef916e696a96 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:40.535405 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 274/541] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:40 2022] GET /identity/v3/role_assignments?user.id=95e7fb8f48dd4348b54e3694a8410bfd&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=716cc96fb28146b7b9ae6271569a206f => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:40.585277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e154d-3dd9-4168-be6e-962a8a750d91 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:40.585564 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e154d-3dd9-4168-be6e-962a8a750d91 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:40.585768 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e154d-3dd9-4168-be6e-962a8a750d91 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:40.587296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 268/542] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:40.592922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f733d6f-37af-49f0-951d-a16b04e0af60 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:40.593218 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f733d6f-37af-49f0-951d-a16b04e0af60 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:40.593566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f733d6f-37af-49f0-951d-a16b04e0af60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:40.616809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2f733d6f-37af-49f0-951d-a16b04e0af60 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:40.618020 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2f733d6f-37af-49f0-951d-a16b04e0af60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:40.715205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 275/543] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:40.758575 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e154d-3dd9-4168-be6e-962a8a750d91 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:40.960814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e154d-3dd9-4168-be6e-962a8a750d91 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:41.008222 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d9a9a-c6f6-4c92-a1b6-433b0917d60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:41.009380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d9a9a-c6f6-4c92-a1b6-433b0917d60f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:41.009875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d9a9a-c6f6-4c92-a1b6-433b0917d60f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:41.010370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d9a9a-c6f6-4c92-a1b6-433b0917d60f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:41.026590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0d9a9a-c6f6-4c92-a1b6-433b0917d60f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-3b0d9a9a-c6f6-4c92-a1b6-433b0917d60f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:41.034759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:41.035668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:41.037481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:41.037481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:41.037481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:41.037481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:41.037481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:41.037481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:41.037481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:41.037481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:41.037481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:41.037481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:41.037481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:41.037481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:41.044792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 269/544] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:41.045868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f733d6f-37af-49f0-951d-a16b04e0af60 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:41.206047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f733d6f-37af-49f0-951d-a16b04e0af60 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:41.218473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:41.257360 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb1793-a251-4743-a88e-52b4099812fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:41.258038 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb1793-a251-4743-a88e-52b4099812fa None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:41.258321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb1793-a251-4743-a88e-52b4099812fa None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:41.258597 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb1793-a251-4743-a88e-52b4099812fa None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:41.259373 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb1793-a251-4743-a88e-52b4099812fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:41.259682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb1793-a251-4743-a88e-52b4099812fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:41.261067 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb1793-a251-4743-a88e-52b4099812fa None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:41.275296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 276/545] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:41.280772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d9a9a-c6f6-4c92-a1b6-433b0917d60f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:41.309097 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:41.397620 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe6984-14fe-451b-b05a-c9f37f8cf8f8 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:41.398069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe6984-14fe-451b-b05a-c9f37f8cf8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:41.398463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe6984-14fe-451b-b05a-c9f37f8cf8f8 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:41.400114 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 277/546] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:41.405339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c1d67-5fac-4c73-ba6f-4dffc7cab929 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:41.405632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c1d67-5fac-4c73-ba6f-4dffc7cab929 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:41.405914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c1d67-5fac-4c73-ba6f-4dffc7cab929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:41.411611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d9a9a-c6f6-4c92-a1b6-433b0917d60f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:41.423865 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1f0c1d67-5fac-4c73-ba6f-4dffc7cab929 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:41.425163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1f0c1d67-5fac-4c73-ba6f-4dffc7cab929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:41.446933 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594324ce-9387-4071-84f1-37b0dec9f196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:41.447766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594324ce-9387-4071-84f1-37b0dec9f196 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:41.448045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594324ce-9387-4071-84f1-37b0dec9f196 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:41.448317 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594324ce-9387-4071-84f1-37b0dec9f196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:41.459771 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594324ce-9387-4071-84f1-37b0dec9f196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:41.461216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594324ce-9387-4071-84f1-37b0dec9f196 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:41.478442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-594324ce-9387-4071-84f1-37b0dec9f196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:41.478442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:41.478442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:41.478442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:41.478442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:41.478442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:41.478442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:41.478442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:41.478442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:41.478442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:41.478442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:41.478442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:41.478442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:41.478442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:41.478442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:41.479389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:41.480343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:41.480343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:41.480343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:41.480343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:41.480343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 270/547] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:41.492590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594324ce-9387-4071-84f1-37b0dec9f196 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:41.522464 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 278/548] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 15:00:41.556793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:41.564855 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c1d67-5fac-4c73-ba6f-4dffc7cab929 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:41.595964 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594324ce-9387-4071-84f1-37b0dec9f196 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:41.658298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1bdaf-020f-4bea-88be-fc569a9e3eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:41.659055 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1bdaf-020f-4bea-88be-fc569a9e3eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:41.659262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1bdaf-020f-4bea-88be-fc569a9e3eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:41.659536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1bdaf-020f-4bea-88be-fc569a9e3eb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:41.660262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1bdaf-020f-4bea-88be-fc569a9e3eb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:41.660568 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1bdaf-020f-4bea-88be-fc569a9e3eb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:41.662780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1bdaf-020f-4bea-88be-fc569a9e3eb3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:41.686266 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 271/549] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:41.692956 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1bdaf-020f-4bea-88be-fc569a9e3eb3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:41.747114 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:41.752710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c1d67-5fac-4c73-ba6f-4dffc7cab929 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:41.804616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf8155-8a58-40de-b5f8-786fc1117065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:41.805454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf8155-8a58-40de-b5f8-786fc1117065 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:41.805749 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf8155-8a58-40de-b5f8-786fc1117065 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:41.806033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf8155-8a58-40de-b5f8-786fc1117065 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:41.820569 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf8155-8a58-40de-b5f8-786fc1117065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e4cf8155-8a58-40de-b5f8-786fc1117065 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:41.831154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:41.832190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:41.833903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:41.833903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:41.833903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:41.833903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:41.833903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:41.833903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:41.833903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:41.833903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:41.833903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:41.833903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:41.833903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:41.833903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:41.833903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 279/550] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:41.834704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25e3c0-a9b2-463a-84b3-3e5de66a7438 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:41.840265 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25e3c0-a9b2-463a-84b3-3e5de66a7438 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:41.840265 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25e3c0-a9b2-463a-84b3-3e5de66a7438 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:41.840667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 280/551] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:41.841436 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25e3c0-a9b2-463a-84b3-3e5de66a7438 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:41.845739 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1bdaf-020f-4bea-88be-fc569a9e3eb3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:41.872385 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:41.885528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507cae4-1f11-4fd8-8555-d69ce546faa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:41.886712 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507cae4-1f11-4fd8-8555-d69ce546faa3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:41.887239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507cae4-1f11-4fd8-8555-d69ce546faa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:41.887721 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507cae4-1f11-4fd8-8555-d69ce546faa3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:41.897061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507cae4-1f11-4fd8-8555-d69ce546faa3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:41.898803 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507cae4-1f11-4fd8-8555-d69ce546faa3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-9507cae4-1f11-4fd8-8555-d69ce546faa3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:41.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:41.919852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:41.920782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:41.920782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:41.920782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:41.920782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:41.920782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:41.920782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:41.920782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:41.920782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:41.920782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 15:00:41.920782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:41.927604 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 272/552] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:00:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:41.929559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21842a-923b-416d-93c6-7339562f0eed None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:41.930073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21842a-923b-416d-93c6-7339562f0eed None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:41.930558 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21842a-923b-416d-93c6-7339562f0eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:41.951516 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-fb21842a-923b-416d-93c6-7339562f0eed None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:41.953138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fb21842a-923b-416d-93c6-7339562f0eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:41.979334 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25e3c0-a9b2-463a-84b3-3e5de66a7438 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:42.030149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 273/553] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:42.031407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21842a-923b-416d-93c6-7339562f0eed None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:42.035964 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e336d-4d17-4259-aa73-459df628bbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:42.037864 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e336d-4d17-4259-aa73-459df628bbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:42.039521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e336d-4d17-4259-aa73-459df628bbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:42.040008 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e336d-4d17-4259-aa73-459df628bbe9 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:42.044044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e336d-4d17-4259-aa73-459df628bbe9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:42.044598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e336d-4d17-4259-aa73-459df628bbe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:42.046344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e336d-4d17-4259-aa73-459df628bbe9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:42.065980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 281/554] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:42.065980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e336d-4d17-4259-aa73-459df628bbe9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:42.097321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:42.171017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21842a-923b-416d-93c6-7339562f0eed None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:42.186886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:42.199016 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e336d-4d17-4259-aa73-459df628bbe9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:42.212375 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757eab6-71a4-4cef-8e73-aef1de2513a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:42.213479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757eab6-71a4-4cef-8e73-aef1de2513a1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:42.214173 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757eab6-71a4-4cef-8e73-aef1de2513a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:42.214692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757eab6-71a4-4cef-8e73-aef1de2513a1 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:42.215805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757eab6-71a4-4cef-8e73-aef1de2513a1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:42.216381 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757eab6-71a4-4cef-8e73-aef1de2513a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:42.218243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757eab6-71a4-4cef-8e73-aef1de2513a1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:42.241689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bc08e-7ef2-409a-9fbd-8eb6b35d1a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:42.243216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 274/555] 10.222.0.67 () {56 vars in 1135 bytes} [Fri Jun 24 15:00:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:42.243216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757eab6-71a4-4cef-8e73-aef1de2513a1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:42.247547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bc08e-7ef2-409a-9fbd-8eb6b35d1a3c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:42.248065 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bc08e-7ef2-409a-9fbd-8eb6b35d1a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:42.248362 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4bc08e-7ef2-409a-9fbd-8eb6b35d1a3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:42.260502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4bc08e-7ef2-409a-9fbd-8eb6b35d1a3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8f4bc08e-7ef2-409a-9fbd-8eb6b35d1a3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:42.268644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:42.269596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:42.271371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:42.271371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:42.271371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:42.271371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:42.271371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:42.271371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:42.271371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:42.271371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:42.271371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:42.271371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:42.271371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:42.271371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:42.271371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 282/556] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:42.272352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bc08e-7ef2-409a-9fbd-8eb6b35d1a3c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:42.361001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757eab6-71a4-4cef-8e73-aef1de2513a1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:42.368311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:42.387307 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4bc08e-7ef2-409a-9fbd-8eb6b35d1a3c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:42.399113 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff80b67-2987-4ba9-9a8e-f7e01abbd1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:42.399962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff80b67-2987-4ba9-9a8e-f7e01abbd1ef None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:42.400251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff80b67-2987-4ba9-9a8e-f7e01abbd1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:42.400532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff80b67-2987-4ba9-9a8e-f7e01abbd1ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:42.411361 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff80b67-2987-4ba9-9a8e-f7e01abbd1ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:42.412818 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff80b67-2987-4ba9-9a8e-f7e01abbd1ef None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:42.428262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7a194-0e11-4076-a7ff-879df45d839d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:42.429101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7a194-0e11-4076-a7ff-879df45d839d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:42.429557 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7a194-0e11-4076-a7ff-879df45d839d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:42.429978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7a194-0e11-4076-a7ff-879df45d839d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:42.431723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-eff80b67-2987-4ba9-9a8e-f7e01abbd1ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:42.431723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:42.431723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:42.431723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:42.431723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:42.431723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:42.431723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:42.431723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:42.431723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:42.431723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:42.431723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:42.431723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:42.431723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:42.431723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:42.431723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:42.433092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:42.433995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:42.433995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:42.433995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:42.433995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:42.433995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 275/557] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:42.433995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff80b67-2987-4ba9-9a8e-f7e01abbd1ef None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:42.463426 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:42.508818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7a194-0e11-4076-a7ff-879df45d839d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:42.510658 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7a194-0e11-4076-a7ff-879df45d839d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:42.549175 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff80b67-2987-4ba9-9a8e-f7e01abbd1ef None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:42.588913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a952df-6130-44d3-b61f-0c24ae3de2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:42.589903 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a952df-6130-44d3-b61f-0c24ae3de2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:42.590187 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a952df-6130-44d3-b61f-0c24ae3de2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:42.590465 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a952df-6130-44d3-b61f-0c24ae3de2c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:42.591229 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a952df-6130-44d3-b61f-0c24ae3de2c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:42.591538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a952df-6130-44d3-b61f-0c24ae3de2c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:42.592910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a952df-6130-44d3-b61f-0c24ae3de2c4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:42.596203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:42.601988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 276/558] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:42.604608 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c31b4-91fa-4743-af8b-b4cfc9f4c0be None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:42.604898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c31b4-91fa-4743-af8b-b4cfc9f4c0be None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:42.605177 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c31b4-91fa-4743-af8b-b4cfc9f4c0be None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:42.610616 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 277/559] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:42.610616 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c31b4-91fa-4743-af8b-b4cfc9f4c0be None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:42.620747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 283/560] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:42.621379 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7a194-0e11-4076-a7ff-879df45d839d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:42.707730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:42.726725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7a194-0e11-4076-a7ff-879df45d839d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:42.726897 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c31b4-91fa-4743-af8b-b4cfc9f4c0be None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:42.761977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42635b0-6f1b-495b-be28-1140cff90d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:42.762779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42635b0-6f1b-495b-be28-1140cff90d8a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:42.763059 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42635b0-6f1b-495b-be28-1140cff90d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:42.763363 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42635b0-6f1b-495b-be28-1140cff90d8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:42.764243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72470e1-549c-4cca-ab0d-3b46bf9e6018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:42.764912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72470e1-549c-4cca-ab0d-3b46bf9e6018 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:42.765373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72470e1-549c-4cca-ab0d-3b46bf9e6018 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:42.765660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72470e1-549c-4cca-ab0d-3b46bf9e6018 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:42.766719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72470e1-549c-4cca-ab0d-3b46bf9e6018 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:42.767055 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72470e1-549c-4cca-ab0d-3b46bf9e6018 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:42.768531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72470e1-549c-4cca-ab0d-3b46bf9e6018 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:42.773404 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42635b0-6f1b-495b-be28-1140cff90d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:42.774835 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42635b0-6f1b-495b-be28-1140cff90d8a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:42.787986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 278/561] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:42.789317 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e58288-ea7b-4398-9563-5b939e3f4834 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:42.789606 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e58288-ea7b-4398-9563-5b939e3f4834 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:42.789883 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e58288-ea7b-4398-9563-5b939e3f4834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:42.797378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e42635b0-6f1b-495b-be28-1140cff90d8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:42.797378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:42.797378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:42.797378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:42.797378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:42.797378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:42.797378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:42.797378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:42.797378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:42.797378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:42.797378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:42.797378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:42.797378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:42.797378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:42.797378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:42.798311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:42.799310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:42.799310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:42.799310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:42.799310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:42.799310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 284/562] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:42.799765 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42635b0-6f1b-495b-be28-1140cff90d8a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:42.813673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-45e58288-ea7b-4398-9563-5b939e3f4834 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:42.815026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-45e58288-ea7b-4398-9563-5b939e3f4834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:42.836224 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:42.887755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 279/563] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:42.896080 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e58288-ea7b-4398-9563-5b939e3f4834 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:42.908188 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42635b0-6f1b-495b-be28-1140cff90d8a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:42.943801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0bf62-5c9c-4310-8b5a-7713bbaeac22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:42.944602 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0bf62-5c9c-4310-8b5a-7713bbaeac22 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:42.944889 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0bf62-5c9c-4310-8b5a-7713bbaeac22 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:42.945180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0bf62-5c9c-4310-8b5a-7713bbaeac22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:42.954991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0bf62-5c9c-4310-8b5a-7713bbaeac22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:42.956509 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0bf62-5c9c-4310-8b5a-7713bbaeac22 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:42.970639 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:42.974706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-59b0bf62-5c9c-4310-8b5a-7713bbaeac22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:42.974706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:42.974706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:42.974706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:42.974706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:42.974706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:42.974706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:42.974706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:42.974706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:42.974706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:42.974706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:42.974706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:42.974706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:42.974706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:42.974706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:42.975671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:42.976573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:42.976573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:42.976573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:42.976573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:42.976573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 285/564] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:42.976863 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0bf62-5c9c-4310-8b5a-7713bbaeac22 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:43.000005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e58288-ea7b-4398-9563-5b939e3f4834 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:43.033520 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124aff7d-64fd-4d56-b9cd-f0884151f8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:43.034321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124aff7d-64fd-4d56-b9cd-f0884151f8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:43.034605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124aff7d-64fd-4d56-b9cd-f0884151f8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:43.034884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124aff7d-64fd-4d56-b9cd-f0884151f8a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:43.042445 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124aff7d-64fd-4d56-b9cd-f0884151f8a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:43.043915 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124aff7d-64fd-4d56-b9cd-f0884151f8a6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-124aff7d-64fd-4d56-b9cd-f0884151f8a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:43.058407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:43.059514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:43.061027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:43.061027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:43.061027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:43.061027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:43.061027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:43.061027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:43.061027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:43.061027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:43.061027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:00:43.061027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:43.061027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 280/565] 10.222.0.67 () {56 vars in 1093 bytes} [Fri Jun 24 15:00:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:43.061027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da83db8-9590-42a4-aad3-81cb20d988da None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:43.061850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da83db8-9590-42a4-aad3-81cb20d988da None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:43.061850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da83db8-9590-42a4-aad3-81cb20d988da None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:43.063217 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 281/566] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:43.063335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da83db8-9590-42a4-aad3-81cb20d988da None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:43.064612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:43.081927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0bf62-5c9c-4310-8b5a-7713bbaeac22 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:43.118336 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea8347-af94-467b-9fa1-fd4e57abfaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:43.118997 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea8347-af94-467b-9fa1-fd4e57abfaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:43.119411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea8347-af94-467b-9fa1-fd4e57abfaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:43.119691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea8347-af94-467b-9fa1-fd4e57abfaa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:43.120406 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea8347-af94-467b-9fa1-fd4e57abfaa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:43.120704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea8347-af94-467b-9fa1-fd4e57abfaa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:43.122070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea8347-af94-467b-9fa1-fd4e57abfaa1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:43.132969 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 286/567] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:43.137445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea8347-af94-467b-9fa1-fd4e57abfaa1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:43.164994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da83db8-9590-42a4-aad3-81cb20d988da None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:43.197981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40401ad-e02a-4e46-968d-3458c1dc0bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:43.198833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40401ad-e02a-4e46-968d-3458c1dc0bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:43.199124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40401ad-e02a-4e46-968d-3458c1dc0bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:43.199468 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40401ad-e02a-4e46-968d-3458c1dc0bc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:43.201362 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:43.210723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40401ad-e02a-4e46-968d-3458c1dc0bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a40401ad-e02a-4e46-968d-3458c1dc0bc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:43.218605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:43.219580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:43.221139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:43.221139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:43.221139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:43.221139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:43.221139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:43.221139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:43.221139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:43.221139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:43.221139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:43.221139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:43.221139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:43.221139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:43.221139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 282/568] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:43.221139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40401ad-e02a-4e46-968d-3458c1dc0bc3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:43.234046 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea8347-af94-467b-9fa1-fd4e57abfaa1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:43.266661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bccf2-5265-44c7-9289-d3b831bc81d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:43.267359 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bccf2-5265-44c7-9289-d3b831bc81d5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:43.267647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bccf2-5265-44c7-9289-d3b831bc81d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:43.267928 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bccf2-5265-44c7-9289-d3b831bc81d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:43.268651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bccf2-5265-44c7-9289-d3b831bc81d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:43.268961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bccf2-5265-44c7-9289-d3b831bc81d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:43.270890 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bccf2-5265-44c7-9289-d3b831bc81d5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:43.282382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 287/569] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:43.283662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dda691-85b0-4fee-b8fa-66828aedcc63 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:43.283959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dda691-85b0-4fee-b8fa-66828aedcc63 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:43.284241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dda691-85b0-4fee-b8fa-66828aedcc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:43.296136 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:43.301813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-78dda691-85b0-4fee-b8fa-66828aedcc63 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:43.303282 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-78dda691-85b0-4fee-b8fa-66828aedcc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:43.320226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40401ad-e02a-4e46-968d-3458c1dc0bc3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:43.353895 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484c88-10d1-4d6a-8bfc-685006b996bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:43.354895 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484c88-10d1-4d6a-8bfc-685006b996bd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:43.354895 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484c88-10d1-4d6a-8bfc-685006b996bd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:43.355115 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484c88-10d1-4d6a-8bfc-685006b996bd None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:43.355966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df484c88-10d1-4d6a-8bfc-685006b996bd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:43.356280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df484c88-10d1-4d6a-8bfc-685006b996bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:43.357706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df484c88-10d1-4d6a-8bfc-685006b996bd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:43.367692 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 288/570] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:43.368510 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dda691-85b0-4fee-b8fa-66828aedcc63 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:43.380318 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 283/571] 10.222.0.67 () {56 vars in 1133 bytes} [Fri Jun 24 15:00:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:43.380318 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484c88-10d1-4d6a-8bfc-685006b996bd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:43.431016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:43.469732 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dda691-85b0-4fee-b8fa-66828aedcc63 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:43.477220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484c88-10d1-4d6a-8bfc-685006b996bd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:43.503842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ca4b4-737a-49d3-a6c1-721e08d60f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:43.504705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ca4b4-737a-49d3-a6c1-721e08d60f71 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:43.504990 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ca4b4-737a-49d3-a6c1-721e08d60f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:43.505496 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ca4b4-737a-49d3-a6c1-721e08d60f71 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:43.508075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:43.510486 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05a1e6-8be5-4315-9cf1-2bbe0c71d562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:43.511136 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05a1e6-8be5-4315-9cf1-2bbe0c71d562 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:43.511458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05a1e6-8be5-4315-9cf1-2bbe0c71d562 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:43.511728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05a1e6-8be5-4315-9cf1-2bbe0c71d562 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:43.512460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05a1e6-8be5-4315-9cf1-2bbe0c71d562 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:43.512584 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ca4b4-737a-49d3-a6c1-721e08d60f71 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:43.512761 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05a1e6-8be5-4315-9cf1-2bbe0c71d562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:43.513907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ca4b4-737a-49d3-a6c1-721e08d60f71 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:43.514084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05a1e6-8be5-4315-9cf1-2bbe0c71d562 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:43.522209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 284/572] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:43.525435 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05a1e6-8be5-4315-9cf1-2bbe0c71d562 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-082ca4b4-737a-49d3-a6c1-721e08d60f71 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:43.528446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:43.529383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:43.530431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:43.530431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:43.530431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:43.530431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:43.530431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:43.530431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:43.530431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:43.530431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:43.530431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 15:00:43.530431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:43.530431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 289/573] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:43.530431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ca4b4-737a-49d3-a6c1-721e08d60f71 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:43.623425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05a1e6-8be5-4315-9cf1-2bbe0c71d562 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:43.628678 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ca4b4-737a-49d3-a6c1-721e08d60f71 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:43.637514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:43.657261 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9816e5-1777-49be-9732-9c0196b115a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:43.658079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9816e5-1777-49be-9732-9c0196b115a2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:43.658369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9816e5-1777-49be-9732-9c0196b115a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:43.658652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9816e5-1777-49be-9732-9c0196b115a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:43.663253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81786bc2-2818-471c-9f95-c991d1b7ca69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:43.664033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81786bc2-2818-471c-9f95-c991d1b7ca69 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:43.664315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81786bc2-2818-471c-9f95-c991d1b7ca69 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:43.664588 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81786bc2-2818-471c-9f95-c991d1b7ca69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:43.665624 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9816e5-1777-49be-9732-9c0196b115a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:43.667083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9816e5-1777-49be-9732-9c0196b115a2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:43.674259 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81786bc2-2818-471c-9f95-c991d1b7ca69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:43.675719 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81786bc2-2818-471c-9f95-c991d1b7ca69 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:43.681413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-4e9816e5-1777-49be-9732-9c0196b115a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:43.681413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:43.681413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:43.681413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:43.681413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:43.681413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:43.681413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:43.681413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:43.681413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:43.681413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:43.681413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:43.681413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:43.681413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:43.681413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:43.681413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:43.682343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:43.683696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:43.683696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:43.683696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:43.683696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:43.683696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:43.683696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:43.683696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:43.683696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:43.683696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 285/574] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:43.683696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9816e5-1777-49be-9732-9c0196b115a2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:43.694078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-81786bc2-2818-471c-9f95-c991d1b7ca69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:43.694078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:43.694078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:43.694078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:43.694078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:43.694078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:43.694078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:43.694078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:43.694078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:43.694078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:43.694078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:43.694078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:43.694078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:43.694078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:43.694078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:43.694977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:43.696013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:43.696013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:43.696013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:43.696013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:43.696013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 290/575] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:43.696013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81786bc2-2818-471c-9f95-c991d1b7ca69 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:43.748177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:43.808546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81786bc2-2818-471c-9f95-c991d1b7ca69 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:43.809209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9816e5-1777-49be-9732-9c0196b115a2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:43.841719 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbf507-41d2-424c-8454-21d8a6feee03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:43.842505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbf507-41d2-424c-8454-21d8a6feee03 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:43.842787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbf507-41d2-424c-8454-21d8a6feee03 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:43.843062 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbf507-41d2-424c-8454-21d8a6feee03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:43.843290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9864d-bfdb-4ebe-a547-5805107b7c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:43.844080 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9864d-bfdb-4ebe-a547-5805107b7c09 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:43.844364 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9864d-bfdb-4ebe-a547-5805107b7c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:43.844638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9864d-bfdb-4ebe-a547-5805107b7c09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:43.853839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbf507-41d2-424c-8454-21d8a6feee03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:43.855302 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbf507-41d2-424c-8454-21d8a6feee03 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:43.855425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9864d-bfdb-4ebe-a547-5805107b7c09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-acc9864d-bfdb-4ebe-a547-5805107b7c09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:43.863014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:43.864093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:43.865041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:43.865041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:43.865041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:43.865041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:43.865041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:43.865041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:43.865041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:43.865041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:43.865041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:43.865041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:43.865041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:43.865041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:43.865041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 286/576] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:43.865041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9864d-bfdb-4ebe-a547-5805107b7c09 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:43.876409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a8dbf507-41d2-424c-8454-21d8a6feee03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:43.876409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:43.876409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:43.876409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:43.876409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:43.876409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:43.876409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:43.876409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:43.876409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:43.876409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:43.876409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:43.876409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:43.876409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:43.876409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:43.876409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:43.877396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:43.878497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:43.878497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:43.878497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:43.878497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:43.878497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 291/577] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:43.878497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbf507-41d2-424c-8454-21d8a6feee03 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:43.900625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:43.966863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9864d-bfdb-4ebe-a547-5805107b7c09 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:43.976396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:43.979040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbf507-41d2-424c-8454-21d8a6feee03 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:43.999952 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00735db8-f716-457f-9136-d0a813f54471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:44.000612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00735db8-f716-457f-9136-d0a813f54471 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:44.000894 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00735db8-f716-457f-9136-d0a813f54471 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:44.001163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00735db8-f716-457f-9136-d0a813f54471 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:44.001987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00735db8-f716-457f-9136-d0a813f54471 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:44.002288 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00735db8-f716-457f-9136-d0a813f54471 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:44.003747 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00735db8-f716-457f-9136-d0a813f54471 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:44.011776 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f2b4b6-ac65-4af7-aff9-dbdd2e6d6f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:44.012437 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2b4b6-ac65-4af7-aff9-dbdd2e6d6f0d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:44.012721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2b4b6-ac65-4af7-aff9-dbdd2e6d6f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:44.013004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2b4b6-ac65-4af7-aff9-dbdd2e6d6f0d None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:44.013789 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f2b4b6-ac65-4af7-aff9-dbdd2e6d6f0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:44.014099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f2b4b6-ac65-4af7-aff9-dbdd2e6d6f0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:44.015539 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f2b4b6-ac65-4af7-aff9-dbdd2e6d6f0d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:44.021670 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 287/578] 10.222.0.67 () {56 vars in 1137 bytes} [Fri Jun 24 15:00:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:44.022468 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00735db8-f716-457f-9136-d0a813f54471 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:44.032337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 292/579] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:44.032337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f2b4b6-ac65-4af7-aff9-dbdd2e6d6f0d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:44.115591 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:44.123851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00735db8-f716-457f-9136-d0a813f54471 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:44.133642 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f2b4b6-ac65-4af7-aff9-dbdd2e6d6f0d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:44.160232 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760760e-2360-473b-8bcd-abe3c2b02267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:44.161039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760760e-2360-473b-8bcd-abe3c2b02267 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:44.161482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760760e-2360-473b-8bcd-abe3c2b02267 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:44.161764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760760e-2360-473b-8bcd-abe3c2b02267 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:44.162492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760760e-2360-473b-8bcd-abe3c2b02267 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:44.162801 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760760e-2360-473b-8bcd-abe3c2b02267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:44.164207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760760e-2360-473b-8bcd-abe3c2b02267 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:44.166930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5a4d8-a313-45dc-a232-36a3d52963c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:44.168174 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5a4d8-a313-45dc-a232-36a3d52963c9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:44.168174 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5a4d8-a313-45dc-a232-36a3d52963c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:44.169218 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5a4d8-a313-45dc-a232-36a3d52963c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:44.169897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5a4d8-a313-45dc-a232-36a3d52963c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:44.170206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5a4d8-a313-45dc-a232-36a3d52963c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:44.171707 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5a4d8-a313-45dc-a232-36a3d52963c9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:44.175324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 288/580] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:44.180315 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760760e-2360-473b-8bcd-abe3c2b02267 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:44.183807 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 293/581] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:44.184635 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5a4d8-a313-45dc-a232-36a3d52963c9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:44.210273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:44.277223 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760760e-2360-473b-8bcd-abe3c2b02267 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:44.284397 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5a4d8-a313-45dc-a232-36a3d52963c9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:44.310598 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ca795-bad2-4679-a58c-117125e40323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:44.311282 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ca795-bad2-4679-a58c-117125e40323 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:44.311569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ca795-bad2-4679-a58c-117125e40323 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:44.311848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ca795-bad2-4679-a58c-117125e40323 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:44.312557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ca795-bad2-4679-a58c-117125e40323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:44.312865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ca795-bad2-4679-a58c-117125e40323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:44.314221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ca795-bad2-4679-a58c-117125e40323 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:44.319204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a612a-da5c-4485-8ac4-ef414cfd0551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:44.319995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a612a-da5c-4485-8ac4-ef414cfd0551 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:44.320279 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a612a-da5c-4485-8ac4-ef414cfd0551 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:44.320559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a612a-da5c-4485-8ac4-ef414cfd0551 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:44.325609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 289/582] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:44.326232 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ca795-bad2-4679-a58c-117125e40323 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:44.333514 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a612a-da5c-4485-8ac4-ef414cfd0551 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:44.334966 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a612a-da5c-4485-8ac4-ef414cfd0551 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:44.340117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:44.352770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e58a612a-da5c-4485-8ac4-ef414cfd0551 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:44.352770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:44.352770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:44.352770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:44.352770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:44.352770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:44.352770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:44.352770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:44.352770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:44.352770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:44.352770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:44.352770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:44.352770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:44.352770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:44.352770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:44.353833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:44.354756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:44.354756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:44.354756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:44.354756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:44.354756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 294/583] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:44.354756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a612a-da5c-4485-8ac4-ef414cfd0551 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:44.428095 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ca795-bad2-4679-a58c-117125e40323 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:44.429884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:44.453534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a612a-da5c-4485-8ac4-ef414cfd0551 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:44.462077 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b5514-bd84-4d92-a973-03ec2132b1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:44.462730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b5514-bd84-4d92-a973-03ec2132b1fb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:44.463010 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b5514-bd84-4d92-a973-03ec2132b1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:44.463323 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b5514-bd84-4d92-a973-03ec2132b1fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:44.464140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b5514-bd84-4d92-a973-03ec2132b1fb None admin] RBAC: Adding query filter params (role.id=716cc96fb28146b7b9ae6271569a206f, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=e4732ab3c8c446958092246d16b4b4f0) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:44.464448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b5514-bd84-4d92-a973-03ec2132b1fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:44.465863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b5514-bd84-4d92-a973-03ec2132b1fb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:44.471355 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 290/584] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:44 2022] GET /identity/v3/role_assignments?user.id=e4732ab3c8c446958092246d16b4b4f0&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=716cc96fb28146b7b9ae6271569a206f => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:44.472116 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b5514-bd84-4d92-a973-03ec2132b1fb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:44.491639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c77e0f-d072-4282-94a5-c8bd94d04d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:44.492467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c77e0f-d072-4282-94a5-c8bd94d04d45 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:44.492746 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c77e0f-d072-4282-94a5-c8bd94d04d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:44.493034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c77e0f-d072-4282-94a5-c8bd94d04d45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:44.500720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c77e0f-d072-4282-94a5-c8bd94d04d45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:44.502191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c77e0f-d072-4282-94a5-c8bd94d04d45 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:44.517267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-90c77e0f-d072-4282-94a5-c8bd94d04d45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:44.517267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:44.517267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:44.517267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:44.517267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:44.517267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:44.517267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:44.517267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:44.517267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:44.517267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:44.517267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:44.517267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:44.517267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:44.517267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:44.517267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:44.518261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:44.519159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:44.519159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:44.519159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:44.519159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:44.519159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:44.519159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:44.519159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:44.519159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:44.519159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 295/585] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:44.520862 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c77e0f-d072-4282-94a5-c8bd94d04d45 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:44.588042 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:44.619656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b5514-bd84-4d92-a973-03ec2132b1fb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:44.635064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c77e0f-d072-4282-94a5-c8bd94d04d45 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:44.659312 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8fa68-ddcf-40f3-9d5b-31515bf22564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:44.660112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8fa68-ddcf-40f3-9d5b-31515bf22564 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:44.660396 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8fa68-ddcf-40f3-9d5b-31515bf22564 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:44.660740 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8fa68-ddcf-40f3-9d5b-31515bf22564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:44.670368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9537d1-4758-44c4-bba4-fc159735d303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:44.670922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9537d1-4758-44c4-bba4-fc159735d303 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:44.671228 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9537d1-4758-44c4-bba4-fc159735d303 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:44.671508 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9537d1-4758-44c4-bba4-fc159735d303 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:44.672658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8fa68-ddcf-40f3-9d5b-31515bf22564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:44.673532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:44.674104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8fa68-ddcf-40f3-9d5b-31515bf22564 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:44.680251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9537d1-4758-44c4-bba4-fc159735d303 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:44.681687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9537d1-4758-44c4-bba4-fc159735d303 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:44.692226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-64a8fa68-ddcf-40f3-9d5b-31515bf22564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:44.692226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:44.692226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:44.692226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:44.692226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:44.692226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:44.692226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:44.692226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:44.692226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:44.692226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:44.692226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:44.692226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:44.692226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:44.692226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:44.692226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:44.693238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:44.699566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:44.699566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:44.699566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:44.699566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:44.699566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 291/586] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:44.699566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8fa68-ddcf-40f3-9d5b-31515bf22564 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-2c9537d1-4758-44c4-bba4-fc159735d303 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:44.699945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:44.700836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:44.702760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:44.702760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:44.702760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:44.702760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:44.702760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:44.702760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:44.702760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:44.702760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:44.702760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 15:00:44.702760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:44.702760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 296/587] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:44.702760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9537d1-4758-44c4-bba4-fc159735d303 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:44.809791 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9537d1-4758-44c4-bba4-fc159735d303 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:44.814610 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8fa68-ddcf-40f3-9d5b-31515bf22564 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:44.843219 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:44.845224 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea2a2c-3f94-460a-929d-bab60237e05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:44.845990 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea2a2c-3f94-460a-929d-bab60237e05d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:44.846382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea2a2c-3f94-460a-929d-bab60237e05d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:44.846730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea2a2c-3f94-460a-929d-bab60237e05d None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:44.847622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea2a2c-3f94-460a-929d-bab60237e05d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:44.848001 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea2a2c-3f94-460a-929d-bab60237e05d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:44.850152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea2a2c-3f94-460a-929d-bab60237e05d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:44.851648 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb28e2-b37f-4eac-9d10-609ae21aa971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:44.852313 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb28e2-b37f-4eac-9d10-609ae21aa971 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:44.852599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb28e2-b37f-4eac-9d10-609ae21aa971 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:44.852874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb28e2-b37f-4eac-9d10-609ae21aa971 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:44.853595 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb28e2-b37f-4eac-9d10-609ae21aa971 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:44.853898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb28e2-b37f-4eac-9d10-609ae21aa971 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:44.855267 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb28e2-b37f-4eac-9d10-609ae21aa971 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:44.868097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 297/588] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:44.868097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea2a2c-3f94-460a-929d-bab60237e05d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:44.875402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 292/589] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:44.875402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb28e2-b37f-4eac-9d10-609ae21aa971 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:44.924401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:44.973283 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea2a2c-3f94-460a-929d-bab60237e05d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:44.982486 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb28e2-b37f-4eac-9d10-609ae21aa971 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:45.008316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea194ba5-bed9-40f9-b3da-d3a567d6c50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:45.009069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea194ba5-bed9-40f9-b3da-d3a567d6c50f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:45.009537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea194ba5-bed9-40f9-b3da-d3a567d6c50f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:45.009883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea194ba5-bed9-40f9-b3da-d3a567d6c50f None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:45.010787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea194ba5-bed9-40f9-b3da-d3a567d6c50f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:45.011169 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea194ba5-bed9-40f9-b3da-d3a567d6c50f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:45.012758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea194ba5-bed9-40f9-b3da-d3a567d6c50f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:45.018085 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0607733-175c-447b-9abd-0d32d4c58bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:45.018753 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0607733-175c-447b-9abd-0d32d4c58bda None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:45.019037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0607733-175c-447b-9abd-0d32d4c58bda None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:45.019341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0607733-175c-447b-9abd-0d32d4c58bda None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:45.020726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0607733-175c-447b-9abd-0d32d4c58bda None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:45.021039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0607733-175c-447b-9abd-0d32d4c58bda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:45.022445 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0607733-175c-447b-9abd-0d32d4c58bda None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:45.027443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 298/590] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:45.028435 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea194ba5-bed9-40f9-b3da-d3a567d6c50f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:45.043272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 293/591] 10.222.0.67 () {56 vars in 1137 bytes} [Fri Jun 24 15:00:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:45.044094 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0607733-175c-447b-9abd-0d32d4c58bda None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:45.098314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:45.141121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea194ba5-bed9-40f9-b3da-d3a567d6c50f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:45.157541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:45.160005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0607733-175c-447b-9abd-0d32d4c58bda None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:45.180290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2935d-d915-45e4-be86-c658ef2be9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:45.181545 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2935d-d915-45e4-be86-c658ef2be9f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:45.182024 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2935d-d915-45e4-be86-c658ef2be9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:45.183120 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2935d-d915-45e4-be86-c658ef2be9f6 None admin] PATH_INFO: `/v3/projects/06da054e8b7545dfa56780ab56585be8/users/51de59caa3144a1cbf12b20b15e2134f/roles/716cc96fb28146b7b9ae6271569a206f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:45.196986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc6369-2880-472c-9f23-708168e95f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:45.197783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc6369-2880-472c-9f23-708168e95f0a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:45.198068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc6369-2880-472c-9f23-708168e95f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:45.198350 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc6369-2880-472c-9f23-708168e95f0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:45.199864 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2935d-d915-45e4-be86-c658ef2be9f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=06da054e8b7545dfa56780ab56585be8, user_id=51de59caa3144a1cbf12b20b15e2134f, role_id=716cc96fb28146b7b9ae6271569a206f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:45.201410 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2935d-d915-45e4-be86-c658ef2be9f6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:45.205523 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc6369-2880-472c-9f23-708168e95f0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:45.206848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc6369-2880-472c-9f23-708168e95f0a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:45.220588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a2cc6369-2880-472c-9f23-708168e95f0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:45.220588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:45.220588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:45.220588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:45.220588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:45.220588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:45.220588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:45.220588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:45.220588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:45.220588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:45.220588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:45.220588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:45.220588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:45.220588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:45.220588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:45.221655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:45.222644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:45.222644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:45.222644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:45.222644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:45.222644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:45.222644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:45.222644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:45.222644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:45.222644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 294/592] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:45.227554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc6369-2880-472c-9f23-708168e95f0a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:45.255548 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 299/593] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:00:45 2022] PUT /identity/v3/projects/06da054e8b7545dfa56780ab56585be8/users/51de59caa3144a1cbf12b20b15e2134f/roles/716cc96fb28146b7b9ae6271569a206f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:00:45.255548 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2935d-d915-45e4-be86-c658ef2be9f6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:45.361397 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc6369-2880-472c-9f23-708168e95f0a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:45.370256 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:45.401441 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af070d-72be-4e2e-8c82-a6aaaa482720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:45.403247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af070d-72be-4e2e-8c82-a6aaaa482720 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:45.403352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2935d-d915-45e4-be86-c658ef2be9f6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:45.403446 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af070d-72be-4e2e-8c82-a6aaaa482720 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:45.403446 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af070d-72be-4e2e-8c82-a6aaaa482720 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:45.412386 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af070d-72be-4e2e-8c82-a6aaaa482720 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:45.414071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af070d-72be-4e2e-8c82-a6aaaa482720 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-57af070d-72be-4e2e-8c82-a6aaaa482720 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:45.430421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:45.431618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:45.432735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:45.432735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:45.432735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:45.432735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:45.432735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:45.432735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:45.432735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:45.432735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:45.432735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 15:00:45.432735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:45.434986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 295/594] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:45.435933 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af070d-72be-4e2e-8c82-a6aaaa482720 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:45.451501 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084edde9-411a-4578-8bea-20340f7e7877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:45.451501 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084edde9-411a-4578-8bea-20340f7e7877 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:45.451501 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084edde9-411a-4578-8bea-20340f7e7877 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:45.451501 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084edde9-411a-4578-8bea-20340f7e7877 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:45.461965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084edde9-411a-4578-8bea-20340f7e7877 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:45.463560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084edde9-411a-4578-8bea-20340f7e7877 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:45.481978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-084edde9-411a-4578-8bea-20340f7e7877 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:45.481978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:45.481978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:45.481978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:45.481978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:45.481978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:45.481978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:45.481978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:45.481978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:45.481978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:45.481978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:45.481978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:45.481978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:45.481978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:45.481978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:45.482920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:45.484196 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:45.484196 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:45.484196 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:45.484196 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:45.484196 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 300/595] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:45.484196 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084edde9-411a-4578-8bea-20340f7e7877 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:45.503285 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:45.562055 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af070d-72be-4e2e-8c82-a6aaaa482720 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:45.598774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084edde9-411a-4578-8bea-20340f7e7877 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:45.606559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20087df7-5976-4164-828f-2d0c421be0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:45.607810 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20087df7-5976-4164-828f-2d0c421be0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:45.608111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20087df7-5976-4164-828f-2d0c421be0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:45.608394 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20087df7-5976-4164-828f-2d0c421be0b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:45.609200 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20087df7-5976-4164-828f-2d0c421be0b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:45.609512 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20087df7-5976-4164-828f-2d0c421be0b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:45.611350 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20087df7-5976-4164-828f-2d0c421be0b2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:45.631365 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 296/596] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:45.631365 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20087df7-5976-4164-828f-2d0c421be0b2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:45.641630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121926b-a996-4c62-ab0e-16fbdd4cb5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:45.642413 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121926b-a996-4c62-ab0e-16fbdd4cb5ca None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:45.642708 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121926b-a996-4c62-ab0e-16fbdd4cb5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:45.642988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121926b-a996-4c62-ab0e-16fbdd4cb5ca None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:45.643976 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121926b-a996-4c62-ab0e-16fbdd4cb5ca None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:45.644303 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121926b-a996-4c62-ab0e-16fbdd4cb5ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:45.646479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121926b-a996-4c62-ab0e-16fbdd4cb5ca None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:45.663090 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:45.671428 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 301/597] 10.222.0.67 () {56 vars in 1139 bytes} [Fri Jun 24 15:00:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:45.672491 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121926b-a996-4c62-ab0e-16fbdd4cb5ca None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:45.756598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:45.757792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20087df7-5976-4164-828f-2d0c421be0b2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:45.785458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121926b-a996-4c62-ab0e-16fbdd4cb5ca None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:45.803391 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c9aa8-143f-4a82-bb30-84ea55b55fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:45.804160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c9aa8-143f-4a82-bb30-84ea55b55fbd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:45.804445 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c9aa8-143f-4a82-bb30-84ea55b55fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:45.804723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c9aa8-143f-4a82-bb30-84ea55b55fbd None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:45.805454 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c9aa8-143f-4a82-bb30-84ea55b55fbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:45.805761 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c9aa8-143f-4a82-bb30-84ea55b55fbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:45.807392 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c9aa8-143f-4a82-bb30-84ea55b55fbd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:45.824247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 297/598] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:45.824247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c9aa8-143f-4a82-bb30-84ea55b55fbd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:45.829382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087c972-e1b9-4fc7-8ab2-852f202dd6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:45.830617 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087c972-e1b9-4fc7-8ab2-852f202dd6c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:45.831080 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087c972-e1b9-4fc7-8ab2-852f202dd6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:45.831579 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087c972-e1b9-4fc7-8ab2-852f202dd6c0 None admin] PATH_INFO: `/v3/projects/06da054e8b7545dfa56780ab56585be8/users/1971c464293f426f97d6cb199e48193b/roles/716cc96fb28146b7b9ae6271569a206f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:45.848564 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087c972-e1b9-4fc7-8ab2-852f202dd6c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=06da054e8b7545dfa56780ab56585be8, user_id=1971c464293f426f97d6cb199e48193b, role_id=716cc96fb28146b7b9ae6271569a206f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:45.850291 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087c972-e1b9-4fc7-8ab2-852f202dd6c0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:45.910312 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:45.921153 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 302/599] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:00:45 2022] PUT /identity/v3/projects/06da054e8b7545dfa56780ab56585be8/users/1971c464293f426f97d6cb199e48193b/roles/716cc96fb28146b7b9ae6271569a206f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:00:45.922492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087c972-e1b9-4fc7-8ab2-852f202dd6c0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:45.939331 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c9aa8-143f-4a82-bb30-84ea55b55fbd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:46.018162 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4e84d-9909-4781-af84-99c48a7af061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:46.018975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4e84d-9909-4781-af84-99c48a7af061 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:46.019292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4e84d-9909-4781-af84-99c48a7af061 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:46.019571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4e84d-9909-4781-af84-99c48a7af061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:46.046494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4e84d-9909-4781-af84-99c48a7af061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:46.048295 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4e84d-9909-4781-af84-99c48a7af061 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:46.088914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087c972-e1b9-4fc7-8ab2-852f202dd6c0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:46.096451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-76c4e84d-9909-4781-af84-99c48a7af061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:46.096451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:46.096451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:46.096451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:46.096451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:46.096451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:46.096451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:46.096451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:46.096451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:46.096451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:46.096451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:46.096451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:46.096451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:46.096451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:46.096451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:46.097613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:46.103463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:46.103463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:46.103463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:46.103463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:46.103463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 298/600] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:46.106284 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4e84d-9909-4781-af84-99c48a7af061 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:46.139338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:46.155452 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2c546-7552-4d22-a7fe-325acfaf9bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:46.156778 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2c546-7552-4d22-a7fe-325acfaf9bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:46.157663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2c546-7552-4d22-a7fe-325acfaf9bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:46.158097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2c546-7552-4d22-a7fe-325acfaf9bc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:46.170968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2c546-7552-4d22-a7fe-325acfaf9bc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:46.172682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2c546-7552-4d22-a7fe-325acfaf9bc4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:46.194782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-67b2c546-7552-4d22-a7fe-325acfaf9bc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:46.194782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:46.194782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:46.194782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:46.194782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:46.194782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:46.194782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:46.194782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:46.194782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:46.194782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:46.194782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:46.194782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:46.194782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:46.194782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:46.194782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:46.195995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:46.197047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:46.197047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:46.197047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:46.197047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:46.197047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:46.197047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:46.197047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:46.197047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:46.199067 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 303/601] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:46.204675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2c546-7552-4d22-a7fe-325acfaf9bc4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:46.249733 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4e84d-9909-4781-af84-99c48a7af061 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:46.328503 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2c546-7552-4d22-a7fe-325acfaf9bc4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:46.356453 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f113a1-ef88-44ec-89cd-da1ca3303dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:46.357376 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f113a1-ef88-44ec-89cd-da1ca3303dab None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:46.357845 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f113a1-ef88-44ec-89cd-da1ca3303dab None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:46.358271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f113a1-ef88-44ec-89cd-da1ca3303dab None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:46.359149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f113a1-ef88-44ec-89cd-da1ca3303dab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:46.359649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f113a1-ef88-44ec-89cd-da1ca3303dab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:46.361160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f113a1-ef88-44ec-89cd-da1ca3303dab None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:46.367650 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9ed73-7a2a-4790-bedb-dc2d4b3c0d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:46.368319 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9ed73-7a2a-4790-bedb-dc2d4b3c0d70 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:46.368606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9ed73-7a2a-4790-bedb-dc2d4b3c0d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:46.368883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9ed73-7a2a-4790-bedb-dc2d4b3c0d70 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:46.370325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9ed73-7a2a-4790-bedb-dc2d4b3c0d70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:46.370641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9ed73-7a2a-4790-bedb-dc2d4b3c0d70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:46.372211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9ed73-7a2a-4790-bedb-dc2d4b3c0d70 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:46.383203 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:46.387584 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 304/602] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:46.392783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 299/603] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:46.395794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9ed73-7a2a-4790-bedb-dc2d4b3c0d70 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:46.396074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f113a1-ef88-44ec-89cd-da1ca3303dab None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:46.450365 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:46.517000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9ed73-7a2a-4790-bedb-dc2d4b3c0d70 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:46.556610 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0925b-ea8d-4215-96b6-2a80a42e0546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:46.557422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0925b-ea8d-4215-96b6-2a80a42e0546 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:46.557709 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0925b-ea8d-4215-96b6-2a80a42e0546 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:46.557995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0925b-ea8d-4215-96b6-2a80a42e0546 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:46.565692 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0925b-ea8d-4215-96b6-2a80a42e0546 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:46.567927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0925b-ea8d-4215-96b6-2a80a42e0546 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:46.584253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-89b0925b-ea8d-4215-96b6-2a80a42e0546 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:46.584253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:46.584253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:46.584253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:46.584253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:46.584253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:46.584253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:46.584253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:46.584253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:46.584253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:46.584253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:46.584253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:46.584253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:46.584253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:46.584253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:46.585216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:46.586150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:46.586150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:46.586150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:46.586150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:46.586150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:46.586150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:46.586150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:46.586150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:46.586150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 305/604] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:46.587278 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490509f9-a571-402a-9dfb-eff7287092f1 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:46.587568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490509f9-a571-402a-9dfb-eff7287092f1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:46.587849 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490509f9-a571-402a-9dfb-eff7287092f1 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:46.589345 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 306/605] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:46.599047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490509f9-a571-402a-9dfb-eff7287092f1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:46.630965 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f113a1-ef88-44ec-89cd-da1ca3303dab None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:46.672966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf337-f633-453a-a1d2-e839f6866959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:46.674041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf337-f633-453a-a1d2-e839f6866959 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:46.674436 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf337-f633-453a-a1d2-e839f6866959 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:46.674907 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf337-f633-453a-a1d2-e839f6866959 None admin] PATH_INFO: `/v3/projects/06da054e8b7545dfa56780ab56585be8/users/2efb120c084a48858d32f97843d76834/roles/716cc96fb28146b7b9ae6271569a206f` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:46.693879 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bf337-f633-453a-a1d2-e839f6866959 None admin] RBAC: Authorizing `identity:create_grant(project_id=06da054e8b7545dfa56780ab56585be8, user_id=2efb120c084a48858d32f97843d76834, role_id=716cc96fb28146b7b9ae6271569a206f)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:46.695549 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bf337-f633-453a-a1d2-e839f6866959 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:46.755000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490509f9-a571-402a-9dfb-eff7287092f1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:46.758279 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:46.792518 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e81bf-a7b7-4fff-be02-4dd4f2b1ae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:46.793445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e81bf-a7b7-4fff-be02-4dd4f2b1ae11 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:46.793871 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e81bf-a7b7-4fff-be02-4dd4f2b1ae11 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:46.794284 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e81bf-a7b7-4fff-be02-4dd4f2b1ae11 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:46.795340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e81bf-a7b7-4fff-be02-4dd4f2b1ae11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:46.795818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e81bf-a7b7-4fff-be02-4dd4f2b1ae11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:46.797521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e81bf-a7b7-4fff-be02-4dd4f2b1ae11 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:46.812617 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 307/607] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:46.814299 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f422d-bed3-4dcf-9c56-656bf1ef836d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:46.814715 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f422d-bed3-4dcf-9c56-656bf1ef836d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:46.815149 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f422d-bed3-4dcf-9c56-656bf1ef836d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:46.820202 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 300/607] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:00:46 2022] PUT /identity/v3/projects/06da054e8b7545dfa56780ab56585be8/users/2efb120c084a48858d32f97843d76834/roles/716cc96fb28146b7b9ae6271569a206f => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:00:46.824019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf337-f633-453a-a1d2-e839f6866959 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:46.874994 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fe5f422d-bed3-4dcf-9c56-656bf1ef836d None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:46.881512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fe5f422d-bed3-4dcf-9c56-656bf1ef836d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:47.018195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:47.099513 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf337-f633-453a-a1d2-e839f6866959 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:47.107200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 308/608] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:47.128588 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f422d-bed3-4dcf-9c56-656bf1ef836d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:47.172470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5168e7d-6263-4a3d-b6a0-dd28ea6c7ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:47.174204 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5168e7d-6263-4a3d-b6a0-dd28ea6c7ac3 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:47.175836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5168e7d-6263-4a3d-b6a0-dd28ea6c7ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:47.177188 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5168e7d-6263-4a3d-b6a0-dd28ea6c7ac3 None admin] PATH_INFO: `/v3/projects/06da054e8b7545dfa56780ab56585be8/users/95e7fb8f48dd4348b54e3694a8410bfd/roles/716cc96fb28146b7b9ae6271569a206f` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:47.212115 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5168e7d-6263-4a3d-b6a0-dd28ea6c7ac3 None admin] RBAC: Authorizing `identity:create_grant(project_id=06da054e8b7545dfa56780ab56585be8, user_id=95e7fb8f48dd4348b54e3694a8410bfd, role_id=716cc96fb28146b7b9ae6271569a206f)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:47.213915 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5168e7d-6263-4a3d-b6a0-dd28ea6c7ac3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:47.271956 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:47.279697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f422d-bed3-4dcf-9c56-656bf1ef836d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:47.295308 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 301/609] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:00:46 2022] PUT /identity/v3/projects/06da054e8b7545dfa56780ab56585be8/users/95e7fb8f48dd4348b54e3694a8410bfd/roles/716cc96fb28146b7b9ae6271569a206f => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:00:47.350336 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054be66c-0a5e-4309-ba94-5a01716ebb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:47.351576 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054be66c-0a5e-4309-ba94-5a01716ebb5b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:47.352072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054be66c-0a5e-4309-ba94-5a01716ebb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:47.352579 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054be66c-0a5e-4309-ba94-5a01716ebb5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:47.377143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054be66c-0a5e-4309-ba94-5a01716ebb5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-054be66c-0a5e-4309-ba94-5a01716ebb5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:47.385516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:47.386549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:47.388513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:47.388513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:47.388513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:47.388513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:47.388513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:47.388513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:47.388513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:47.388513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:47.388513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:47.388513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:47.388513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:47.388513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:47.388513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 309/610] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:47.396334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5168e7d-6263-4a3d-b6a0-dd28ea6c7ac3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:47.552987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5648d-184e-43e7-a8cf-b7f3e129fa37 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:47.553325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5648d-184e-43e7-a8cf-b7f3e129fa37 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:47.553505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5648d-184e-43e7-a8cf-b7f3e129fa37 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:47.555127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 310/611] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:47.560288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6079d-6266-43ae-a956-79c251cfa213 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:47.560581 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6079d-6266-43ae-a956-79c251cfa213 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:47.560865 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6079d-6266-43ae-a956-79c251cfa213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:47.563060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:47.568539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5168e7d-6263-4a3d-b6a0-dd28ea6c7ac3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:47.590873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-91e6079d-6266-43ae-a956-79c251cfa213 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:47.592395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-91e6079d-6266-43ae-a956-79c251cfa213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:47.617151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622b9b7-1788-46dc-a9b1-75081c0a7ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:47.617922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622b9b7-1788-46dc-a9b1-75081c0a7ecc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:47.618207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622b9b7-1788-46dc-a9b1-75081c0a7ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:47.618482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622b9b7-1788-46dc-a9b1-75081c0a7ecc None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:47.619244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622b9b7-1788-46dc-a9b1-75081c0a7ecc None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:47.619550 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622b9b7-1788-46dc-a9b1-75081c0a7ecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:47.620904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622b9b7-1788-46dc-a9b1-75081c0a7ecc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:47.629682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 302/612] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:47.636039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622b9b7-1788-46dc-a9b1-75081c0a7ecc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:47.732119 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 311/613] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:47.764498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6079d-6266-43ae-a956-79c251cfa213 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:47.813088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622b9b7-1788-46dc-a9b1-75081c0a7ecc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:47.876791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1757a-177e-4055-9fea-5340c9327a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:47.877597 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1757a-177e-4055-9fea-5340c9327a67 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:47.877877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1757a-177e-4055-9fea-5340c9327a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:47.878149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1757a-177e-4055-9fea-5340c9327a67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:47.899238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1757a-177e-4055-9fea-5340c9327a67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:47.900691 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1757a-177e-4055-9fea-5340c9327a67 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:47.911542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:47.930961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-93d1757a-177e-4055-9fea-5340c9327a67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:47.930961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:47.930961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:47.930961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:47.930961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:47.930961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:47.930961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:47.930961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:47.930961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:47.930961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:47.930961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:47.930961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:47.930961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:47.930961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:47.930961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:47.931946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:47.932851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:47.932851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:47.932851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:47.932851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:47.933481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 303/614] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:47.936049 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6079d-6266-43ae-a956-79c251cfa213 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:47.939977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1757a-177e-4055-9fea-5340c9327a67 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:47.966712 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:47.990102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692cb19-4327-4aab-bf47-761050a36d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:47.992849 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692cb19-4327-4aab-bf47-761050a36d3d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:47.993134 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692cb19-4327-4aab-bf47-761050a36d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:47.993408 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692cb19-4327-4aab-bf47-761050a36d3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:48.019816 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692cb19-4327-4aab-bf47-761050a36d3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-c692cb19-4327-4aab-bf47-761050a36d3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:48.028372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:48.029505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:48.030367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:48.030367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:48.030367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:48.030367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:48.030367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:48.030367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:48.030367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:48.030367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:48.030367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:48.030367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:48.030367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:48.030367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:48.030367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 312/615] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:48.036487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692cb19-4327-4aab-bf47-761050a36d3d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:48.126263 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1757a-177e-4055-9fea-5340c9327a67 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:48.196947 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a8702-153a-4a7e-a859-9ece0bf763f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:48.197624 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a8702-153a-4a7e-a859-9ece0bf763f2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:48.197911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a8702-153a-4a7e-a859-9ece0bf763f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:48.198186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a8702-153a-4a7e-a859-9ece0bf763f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:48.198921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a8702-153a-4a7e-a859-9ece0bf763f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:48.200992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a8702-153a-4a7e-a859-9ece0bf763f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:48.202300 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a8702-153a-4a7e-a859-9ece0bf763f2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:48.226994 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692cb19-4327-4aab-bf47-761050a36d3d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:48.229812 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 304/616] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:48.230779 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a8702-153a-4a7e-a859-9ece0bf763f2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:48.294688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b4873-e8b2-449a-a578-2ad55166d135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:48.299220 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b4873-e8b2-449a-a578-2ad55166d135 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:48.299220 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b4873-e8b2-449a-a578-2ad55166d135 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:48.299220 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b4873-e8b2-449a-a578-2ad55166d135 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:48.299220 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b4873-e8b2-449a-a578-2ad55166d135 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:48.299220 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b4873-e8b2-449a-a578-2ad55166d135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:48.299220 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b4873-e8b2-449a-a578-2ad55166d135 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:48.309018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 313/617] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:48.314153 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b4873-e8b2-449a-a578-2ad55166d135 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:48.329241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:48.384738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:48.436869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b4873-e8b2-449a-a578-2ad55166d135 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:48.461974 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a8702-153a-4a7e-a859-9ece0bf763f2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:48.492908 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4506f-d978-45a4-9dc1-d75b12e2bf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:48.493719 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4506f-d978-45a4-9dc1-d75b12e2bf0e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:48.494005 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4506f-d978-45a4-9dc1-d75b12e2bf0e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:48.494282 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4506f-d978-45a4-9dc1-d75b12e2bf0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:48.497913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c724b78-99be-4c1b-afb2-963eb75feb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:48.498708 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c724b78-99be-4c1b-afb2-963eb75feb71 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:48.498991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c724b78-99be-4c1b-afb2-963eb75feb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:48.499296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c724b78-99be-4c1b-afb2-963eb75feb71 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:48.506214 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4506f-d978-45a4-9dc1-d75b12e2bf0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:48.507388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c724b78-99be-4c1b-afb2-963eb75feb71 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:48.507702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4506f-d978-45a4-9dc1-d75b12e2bf0e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:48.508814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c724b78-99be-4c1b-afb2-963eb75feb71 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:48.526961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-18d4506f-d978-45a4-9dc1-d75b12e2bf0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:48.526961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:48.526961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:48.526961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:48.526961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:48.526961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:48.526961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:48.526961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:48.526961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:48.526961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:48.526961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:48.526961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:48.526961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:48.526961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:48.526961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:48.527897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:48.529022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:48.529022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:48.529022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:48.529022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-3c724b78-99be-4c1b-afb2-963eb75feb71 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:48.529259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:48.530117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:48.531078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:48.531078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:48.531078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:48.531078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:48.531078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:48.531078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:48.531078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:48.531078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:48.531078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 15:00:48.531078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:48.531687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 314/618] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:48.531758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 305/619] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:00:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:48.535265 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a7b73-ea12-419b-a667-6316b94a5be1 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:48.535265 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a7b73-ea12-419b-a667-6316b94a5be1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:48.535265 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a7b73-ea12-419b-a667-6316b94a5be1 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:48.537092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 315/620] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:48.541098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a7b73-ea12-419b-a667-6316b94a5be1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:48.541498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c724b78-99be-4c1b-afb2-963eb75feb71 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:48.615564 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:48.647450 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c724b78-99be-4c1b-afb2-963eb75feb71 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:48.655598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a7b73-ea12-419b-a667-6316b94a5be1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:48.676922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:48.682861 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8308b67-0178-401f-a34a-751b9d7b8015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:48.683669 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8308b67-0178-401f-a34a-751b9d7b8015 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:48.683959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8308b67-0178-401f-a34a-751b9d7b8015 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:48.684238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8308b67-0178-401f-a34a-751b9d7b8015 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:48.685133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8308b67-0178-401f-a34a-751b9d7b8015 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:48.685480 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8308b67-0178-401f-a34a-751b9d7b8015 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:48.686984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8308b67-0178-401f-a34a-751b9d7b8015 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:48.696198 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1481b-3cce-4ff8-a432-4b7248dfdb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:48.697234 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1481b-3cce-4ff8-a432-4b7248dfdb53 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:48.697737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1481b-3cce-4ff8-a432-4b7248dfdb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:48.698210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1481b-3cce-4ff8-a432-4b7248dfdb53 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:48.699211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1481b-3cce-4ff8-a432-4b7248dfdb53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:48.699795 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1481b-3cce-4ff8-a432-4b7248dfdb53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:48.701380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1481b-3cce-4ff8-a432-4b7248dfdb53 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:48.711454 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 306/621] 10.222.0.67 () {56 vars in 1135 bytes} [Fri Jun 24 15:00:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:48.713149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d53161-bc2b-4923-8eb1-550ff5d02156 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:48.713441 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d53161-bc2b-4923-8eb1-550ff5d02156 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:48.713723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d53161-bc2b-4923-8eb1-550ff5d02156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:48.722213 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 316/622] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:48.723845 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1481b-3cce-4ff8-a432-4b7248dfdb53 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:48.732063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-06d53161-bc2b-4923-8eb1-550ff5d02156 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:48.733273 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-06d53161-bc2b-4923-8eb1-550ff5d02156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:48.815201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 307/623] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:48.815201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d53161-bc2b-4923-8eb1-550ff5d02156 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:48.854999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1481b-3cce-4ff8-a432-4b7248dfdb53 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:48.875569 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:48.898758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b08f13-7c64-42ef-b787-ea814df51ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:48.900194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b08f13-7c64-42ef-b787-ea814df51ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:48.901079 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b08f13-7c64-42ef-b787-ea814df51ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:48.901567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b08f13-7c64-42ef-b787-ea814df51ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:48.917427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b08f13-7c64-42ef-b787-ea814df51ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:48.918247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d53161-bc2b-4923-8eb1-550ff5d02156 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:48.919441 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b08f13-7c64-42ef-b787-ea814df51ff2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:48.940775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:48.944150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e8b08f13-7c64-42ef-b787-ea814df51ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:48.944150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:48.944150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:48.944150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:48.944150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:48.944150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:48.944150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:48.944150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:48.944150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:48.944150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:48.944150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:48.944150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:48.944150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:48.944150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:48.944150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:48.945051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:48.945961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:48.945961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:48.945961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:48.945961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:48.954290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 317/624] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:48.954290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b08f13-7c64-42ef-b787-ea814df51ff2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:48.965919 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b4081-6280-408c-b94f-3e0e60fc2b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:48.966999 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b4081-6280-408c-b94f-3e0e60fc2b6d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:48.967539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b4081-6280-408c-b94f-3e0e60fc2b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:48.968111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b4081-6280-408c-b94f-3e0e60fc2b6d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:48.977680 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b4081-6280-408c-b94f-3e0e60fc2b6d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:48.979804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b4081-6280-408c-b94f-3e0e60fc2b6d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-261b4081-6280-408c-b94f-3e0e60fc2b6d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:48.998483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:49.000330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:49.001290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:49.001290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:49.001290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:49.001290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:49.001290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:49.001290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:49.001290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:49.001290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:49.001290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:00:49.001290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:49.005044 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 308/625] 10.222.0.67 () {56 vars in 1093 bytes} [Fri Jun 24 15:00:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:49.006957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b4081-6280-408c-b94f-3e0e60fc2b6d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:49.063359 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b08f13-7c64-42ef-b787-ea814df51ff2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:49.104872 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250f963-0e63-4bd4-af46-ada4f60e3296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:49.105326 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250f963-0e63-4bd4-af46-ada4f60e3296 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:49.105613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250f963-0e63-4bd4-af46-ada4f60e3296 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:49.105892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250f963-0e63-4bd4-af46-ada4f60e3296 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:49.116742 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250f963-0e63-4bd4-af46-ada4f60e3296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-5250f963-0e63-4bd4-af46-ada4f60e3296 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:49.124626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:49.125603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:49.126463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:49.126463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:49.126463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:49.126463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:49.126463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:49.126463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:49.126463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:49.126463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:49.126463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:49.126463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:49.126463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:49.126463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:49.130809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 318/626] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:49.131869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250f963-0e63-4bd4-af46-ada4f60e3296 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:49.138786 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b4081-6280-408c-b94f-3e0e60fc2b6d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:49.165709 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:49.185630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e6650-23b9-474d-b0df-8aa458fccd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:49.187044 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e6650-23b9-474d-b0df-8aa458fccd9b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:49.187586 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e6650-23b9-474d-b0df-8aa458fccd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:49.188092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e6650-23b9-474d-b0df-8aa458fccd9b None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:49.189020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e6650-23b9-474d-b0df-8aa458fccd9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:49.189531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e6650-23b9-474d-b0df-8aa458fccd9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:49.191901 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e6650-23b9-474d-b0df-8aa458fccd9b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:49.213764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 309/627] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:49.213764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e6650-23b9-474d-b0df-8aa458fccd9b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:49.251616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250f963-0e63-4bd4-af46-ada4f60e3296 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:49.256268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:49.290226 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346e8db-6aeb-4eff-aaa7-5588e7e5fbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:49.290895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346e8db-6aeb-4eff-aaa7-5588e7e5fbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:49.291261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346e8db-6aeb-4eff-aaa7-5588e7e5fbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:49.291479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346e8db-6aeb-4eff-aaa7-5588e7e5fbc7 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:49.292318 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346e8db-6aeb-4eff-aaa7-5588e7e5fbc7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:49.292628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346e8db-6aeb-4eff-aaa7-5588e7e5fbc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:49.294067 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346e8db-6aeb-4eff-aaa7-5588e7e5fbc7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:49.312387 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 319/628] 10.222.0.67 () {56 vars in 1133 bytes} [Fri Jun 24 15:00:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:49.313021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346e8db-6aeb-4eff-aaa7-5588e7e5fbc7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:49.318990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e6650-23b9-474d-b0df-8aa458fccd9b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:49.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d105c76-08c8-4e5f-b83c-595524365193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:49.356867 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d105c76-08c8-4e5f-b83c-595524365193 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:49.357151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d105c76-08c8-4e5f-b83c-595524365193 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:49.357433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d105c76-08c8-4e5f-b83c-595524365193 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:49.358186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d105c76-08c8-4e5f-b83c-595524365193 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:49.358501 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d105c76-08c8-4e5f-b83c-595524365193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:49.359909 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d105c76-08c8-4e5f-b83c-595524365193 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:49.371211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 310/629] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:49.371211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9be8d3-5b2b-495b-bad5-fc0e30839300 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:49.371211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9be8d3-5b2b-495b-bad5-fc0e30839300 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:49.371211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9be8d3-5b2b-495b-bad5-fc0e30839300 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:49.372224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 311/630] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:49.376873 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9be8d3-5b2b-495b-bad5-fc0e30839300 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:49.410624 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:49.423351 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346e8db-6aeb-4eff-aaa7-5588e7e5fbc7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:49.460120 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05feed32-69be-4d2a-a5b1-559c45c47e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:49.460933 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05feed32-69be-4d2a-a5b1-559c45c47e0f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:49.461200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05feed32-69be-4d2a-a5b1-559c45c47e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:49.461473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05feed32-69be-4d2a-a5b1-559c45c47e0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:49.468156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05feed32-69be-4d2a-a5b1-559c45c47e0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:49.469592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05feed32-69be-4d2a-a5b1-559c45c47e0f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:49.483794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-05feed32-69be-4d2a-a5b1-559c45c47e0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:49.483794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:49.483794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:49.483794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:49.483794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:49.483794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:49.483794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:49.483794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:49.483794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:49.483794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:49.483794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:49.483794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:49.483794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:49.483794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:49.483794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:49.484747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:49.485647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:49.485647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:49.485647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:49.485647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:49.485647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:49.485647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:49.485647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:49.485647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:49.485647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 320/631] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:49.486745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05feed32-69be-4d2a-a5b1-559c45c47e0f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:49.494607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9be8d3-5b2b-495b-bad5-fc0e30839300 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:49.496403 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:49.533063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2c6e0-bf10-4820-a280-d7dfc67c3ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:49.533741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2c6e0-bf10-4820-a280-d7dfc67c3ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:49.534029 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2c6e0-bf10-4820-a280-d7dfc67c3ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:49.534527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2c6e0-bf10-4820-a280-d7dfc67c3ea9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:49.590714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05feed32-69be-4d2a-a5b1-559c45c47e0f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:49.618346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f2c6e0-bf10-4820-a280-d7dfc67c3ea9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:49.619991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f2c6e0-bf10-4820-a280-d7dfc67c3ea9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:49.626708 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b6fa7-5692-4142-890c-4d05831e986b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:49.627779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b6fa7-5692-4142-890c-4d05831e986b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:49.628225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b6fa7-5692-4142-890c-4d05831e986b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:49.628757 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b6fa7-5692-4142-890c-4d05831e986b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:49.641004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b6fa7-5692-4142-890c-4d05831e986b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:49.642445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b6fa7-5692-4142-890c-4d05831e986b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:49.651979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:49.661936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-772b6fa7-5692-4142-890c-4d05831e986b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:49.661936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:49.661936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:49.661936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:49.661936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:49.661936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:49.661936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:49.661936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:49.661936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:49.661936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:49.661936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:49.661936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:49.661936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:49.661936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:49.661936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:49.662850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:49.663798 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:49.663798 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:49.663798 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:49.663798 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:49.667515 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 321/632] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:49.668616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65f1c2-dc1c-4c34-aa80-51c6acdcc789 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:49.668903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65f1c2-dc1c-4c34-aa80-51c6acdcc789 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:49.669182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65f1c2-dc1c-4c34-aa80-51c6acdcc789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:49.686139 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ee65f1c2-dc1c-4c34-aa80-51c6acdcc789 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:49.687349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ee65f1c2-dc1c-4c34-aa80-51c6acdcc789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:49.728239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 312/633] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:49.729087 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2c6e0-bf10-4820-a280-d7dfc67c3ea9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:49.733650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:49.753660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 322/634] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:49.755870 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70f859-ea4d-445b-bdbf-a7ee500348bd None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:49.756163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70f859-ea4d-445b-bdbf-a7ee500348bd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:49.756444 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70f859-ea4d-445b-bdbf-a7ee500348bd None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:49.757766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 323/635] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:49.758478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70f859-ea4d-445b-bdbf-a7ee500348bd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:49.834142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2c6e0-bf10-4820-a280-d7dfc67c3ea9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:49.859833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70f859-ea4d-445b-bdbf-a7ee500348bd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:49.867279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387de9b-5bfb-4c33-addc-1fd988bf272a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:49.867940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387de9b-5bfb-4c33-addc-1fd988bf272a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:49.868226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387de9b-5bfb-4c33-addc-1fd988bf272a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:49.868504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387de9b-5bfb-4c33-addc-1fd988bf272a None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:49.869284 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387de9b-5bfb-4c33-addc-1fd988bf272a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:49.869596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387de9b-5bfb-4c33-addc-1fd988bf272a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:49.871024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387de9b-5bfb-4c33-addc-1fd988bf272a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:49.884365 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 313/636] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:49.885154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387de9b-5bfb-4c33-addc-1fd988bf272a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:49.890951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:49.896853 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18aefa5-ac6b-4822-90d1-a40aa70bed26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:49.897516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18aefa5-ac6b-4822-90d1-a40aa70bed26 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:49.897801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18aefa5-ac6b-4822-90d1-a40aa70bed26 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:49.898081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18aefa5-ac6b-4822-90d1-a40aa70bed26 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:49.898800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18aefa5-ac6b-4822-90d1-a40aa70bed26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:49.899108 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18aefa5-ac6b-4822-90d1-a40aa70bed26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:49.900490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18aefa5-ac6b-4822-90d1-a40aa70bed26 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:49.911643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 324/637] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:49.912980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47557c-82eb-47ed-916a-e2316d438841 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:49.913272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47557c-82eb-47ed-916a-e2316d438841 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:49.913548 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47557c-82eb-47ed-916a-e2316d438841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:49.930370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8a47557c-82eb-47ed-916a-e2316d438841 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:49.931579 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8a47557c-82eb-47ed-916a-e2316d438841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:49.950950 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:49.989413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387de9b-5bfb-4c33-addc-1fd988bf272a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:49.997060 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 325/638] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:49.998038 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47557c-82eb-47ed-916a-e2316d438841 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:50.024776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac57cd2-7bd4-4da9-b53f-c4e0f472c66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:50.025573 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac57cd2-7bd4-4da9-b53f-c4e0f472c66d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:50.025883 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac57cd2-7bd4-4da9-b53f-c4e0f472c66d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:50.026160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac57cd2-7bd4-4da9-b53f-c4e0f472c66d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:50.036376 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac57cd2-7bd4-4da9-b53f-c4e0f472c66d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:50.037823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac57cd2-7bd4-4da9-b53f-c4e0f472c66d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:50.056569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-fac57cd2-7bd4-4da9-b53f-c4e0f472c66d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:50.056569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:50.056569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:50.056569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:50.056569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:50.056569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:50.056569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:50.056569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:50.056569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:50.056569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:50.056569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:50.056569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:50.056569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:50.056569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:50.056569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:50.057601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:50.058505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:50.058505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:50.058505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:50.058505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:50.058505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 314/639] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:50.058505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac57cd2-7bd4-4da9-b53f-c4e0f472c66d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:50.110242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47557c-82eb-47ed-916a-e2316d438841 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:50.130467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:50.144070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f8916-5352-487b-a247-c68cca5e319a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:50.145419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f8916-5352-487b-a247-c68cca5e319a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:50.146007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f8916-5352-487b-a247-c68cca5e319a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:50.146300 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f8916-5352-487b-a247-c68cca5e319a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:50.156005 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f8916-5352-487b-a247-c68cca5e319a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:50.162995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac57cd2-7bd4-4da9-b53f-c4e0f472c66d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-dd9f8916-5352-487b-a247-c68cca5e319a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:50.164097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:50.165021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:50.165852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:50.165852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:50.165852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:50.165852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:50.165852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:50.165852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:50.165852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:50.165852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:50.165852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:50.165852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:50.165852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:50.165852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:50.165852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 326/640] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:50.166548 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f8916-5352-487b-a247-c68cca5e319a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:50.178696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:50.198939 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0a374-279e-4bdd-974a-ce42ec028329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:50.199863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0a374-279e-4bdd-974a-ce42ec028329 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:50.200146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0a374-279e-4bdd-974a-ce42ec028329 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:50.200422 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0a374-279e-4bdd-974a-ce42ec028329 None admin] PATH_INFO: `/v3/projects/06da054e8b7545dfa56780ab56585be8/users/e4732ab3c8c446958092246d16b4b4f0/roles/716cc96fb28146b7b9ae6271569a206f` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:50.215973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0a374-279e-4bdd-974a-ce42ec028329 None admin] RBAC: Authorizing `identity:create_grant(project_id=06da054e8b7545dfa56780ab56585be8, user_id=e4732ab3c8c446958092246d16b4b4f0, role_id=716cc96fb28146b7b9ae6271569a206f)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:50.218536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0a374-279e-4bdd-974a-ce42ec028329 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:50.264035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f8916-5352-487b-a247-c68cca5e319a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:50.296281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab37e0-b8ad-432c-9734-4cc4df7b48fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:50.297057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab37e0-b8ad-432c-9734-4cc4df7b48fa None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:50.297336 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab37e0-b8ad-432c-9734-4cc4df7b48fa None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:50.297604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab37e0-b8ad-432c-9734-4cc4df7b48fa None admin] PATH_INFO: `/v3/users/glance` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:50.300211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 315/641] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:00:50 2022] PUT /identity/v3/projects/06da054e8b7545dfa56780ab56585be8/users/e4732ab3c8c446958092246d16b4b4f0/roles/716cc96fb28146b7b9ae6271569a206f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:00:50.301020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0a374-279e-4bdd-974a-ce42ec028329 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:50.306501 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab37e0-b8ad-432c-9734-4cc4df7b48fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:50.308062 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab37e0-b8ad-432c-9734-4cc4df7b48fa None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f1ab37e0-b8ad-432c-9734-4cc4df7b48fa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:50.324685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:50.325868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:50.326858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:50.326858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:50.326858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:50.326858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:50.326858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:50.326858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:50.326858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:50.326858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:50.326858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 15:00:50.326858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:50.326858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 327/642] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:50.326858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab37e0-b8ad-432c-9734-4cc4df7b48fa None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:50.374699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:50.419918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0a374-279e-4bdd-974a-ce42ec028329 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:50.448282 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab37e0-b8ad-432c-9734-4cc4df7b48fa None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:50.465711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036956f-9071-4d78-a43c-8728a5b4e828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:50.466732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036956f-9071-4d78-a43c-8728a5b4e828 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:50.466959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036956f-9071-4d78-a43c-8728a5b4e828 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:50.467258 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036956f-9071-4d78-a43c-8728a5b4e828 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:50.479007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036956f-9071-4d78-a43c-8728a5b4e828 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:50.486303 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b92a3-ac7c-47f0-8f0c-e6bdbc4ff1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-1036956f-9071-4d78-a43c-8728a5b4e828 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:50.486806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:50.487763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:50.489216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:50.489216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:50.489216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:50.489216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:50.489216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:50.489216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:50.489216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:50.489216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:50.489216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:50.489216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:50.489216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:50.489216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:50.489981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b92a3-ac7c-47f0-8f0c-e6bdbc4ff1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:50.489981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b92a3-ac7c-47f0-8f0c-e6bdbc4ff1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:50.489981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b92a3-ac7c-47f0-8f0c-e6bdbc4ff1a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:50.489981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b92a3-ac7c-47f0-8f0c-e6bdbc4ff1a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:50.490217 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 316/643] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:50.490294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b92a3-ac7c-47f0-8f0c-e6bdbc4ff1a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:50.490365 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036956f-9071-4d78-a43c-8728a5b4e828 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:50.490957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b92a3-ac7c-47f0-8f0c-e6bdbc4ff1a9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:50.506872 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 328/644] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:50.508278 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b92a3-ac7c-47f0-8f0c-e6bdbc4ff1a9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:50.516998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:50.594693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036956f-9071-4d78-a43c-8728a5b4e828 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:50.614348 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:50.617294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b92a3-ac7c-47f0-8f0c-e6bdbc4ff1a9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:50.631764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0c3c2-f4ce-4cf2-a069-b5e57aa16d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:50.632440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0c3c2-f4ce-4cf2-a069-b5e57aa16d96 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:50.632722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0c3c2-f4ce-4cf2-a069-b5e57aa16d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:50.633004 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0c3c2-f4ce-4cf2-a069-b5e57aa16d96 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:50.633741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e0c3c2-f4ce-4cf2-a069-b5e57aa16d96 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:50.634045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e0c3c2-f4ce-4cf2-a069-b5e57aa16d96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:50.635478 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e0c3c2-f4ce-4cf2-a069-b5e57aa16d96 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:50.648272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 317/645] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:50.648272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0c3c2-f4ce-4cf2-a069-b5e57aa16d96 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:50.658808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092ccb0-a421-4c6c-b27b-7205b786c24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:50.659625 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092ccb0-a421-4c6c-b27b-7205b786c24c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:50.659974 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092ccb0-a421-4c6c-b27b-7205b786c24c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:50.660254 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092ccb0-a421-4c6c-b27b-7205b786c24c None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:50.661125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092ccb0-a421-4c6c-b27b-7205b786c24c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:50.661436 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092ccb0-a421-4c6c-b27b-7205b786c24c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:50.662875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092ccb0-a421-4c6c-b27b-7205b786c24c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:50.685273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 329/646] 10.222.0.67 () {56 vars in 1137 bytes} [Fri Jun 24 15:00:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:50.685273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092ccb0-a421-4c6c-b27b-7205b786c24c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:50.746491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:50.760349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0c3c2-f4ce-4cf2-a069-b5e57aa16d96 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:50.792094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092ccb0-a421-4c6c-b27b-7205b786c24c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:50.795393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cfc35-a2e7-4590-a18a-da8d81750e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:50.796068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cfc35-a2e7-4590-a18a-da8d81750e89 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:50.796348 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cfc35-a2e7-4590-a18a-da8d81750e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:50.796619 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cfc35-a2e7-4590-a18a-da8d81750e89 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:50.797361 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9cfc35-a2e7-4590-a18a-da8d81750e89 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:50.797674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9cfc35-a2e7-4590-a18a-da8d81750e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:50.799031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9cfc35-a2e7-4590-a18a-da8d81750e89 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:50.812429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 318/647] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:50.812429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cfc35-a2e7-4590-a18a-da8d81750e89 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:50.835096 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3dc14-5ce0-4b2e-a7d2-a10ddfa8d7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:50.836059 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3dc14-5ce0-4b2e-a7d2-a10ddfa8d7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:50.836459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3dc14-5ce0-4b2e-a7d2-a10ddfa8d7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:50.836782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3dc14-5ce0-4b2e-a7d2-a10ddfa8d7d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:50.845069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3dc14-5ce0-4b2e-a7d2-a10ddfa8d7d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:50.846506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3dc14-5ce0-4b2e-a7d2-a10ddfa8d7d0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:50.860717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-37f3dc14-5ce0-4b2e-a7d2-a10ddfa8d7d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:50.860717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:50.860717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:50.860717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:50.860717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:50.860717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:50.860717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:50.860717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:50.860717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:50.860717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:50.860717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:50.860717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:50.860717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:50.860717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:50.860717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:50.862084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:50.862979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:50.862979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:50.862979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:50.862979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:50.862979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:50.862979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:50.862979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:50.862979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:50.862979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 330/648] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:50.862979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3dc14-5ce0-4b2e-a7d2-a10ddfa8d7d0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:50.885214 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:50.922958 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cfc35-a2e7-4590-a18a-da8d81750e89 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:50.961301 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e9999-1067-44e2-97cc-3004cc2e0c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:50.962429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e9999-1067-44e2-97cc-3004cc2e0c67 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:50.962902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e9999-1067-44e2-97cc-3004cc2e0c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:50.963397 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e9999-1067-44e2-97cc-3004cc2e0c67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:50.969672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3dc14-5ce0-4b2e-a7d2-a10ddfa8d7d0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:50.975449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e9999-1067-44e2-97cc-3004cc2e0c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:50.977070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e9999-1067-44e2-97cc-3004cc2e0c67 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:50.986188 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:51.009836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-604e9999-1067-44e2-97cc-3004cc2e0c67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:51.009836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:51.009836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:51.009836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:51.009836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:51.009836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:51.009836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:51.009836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:51.009836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:51.009836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:51.009836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:51.009836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:51.009836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:51.009836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:51.009836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:51.010953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:51.012287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:51.012287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:51.012287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:51.012287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:51.012287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 319/649] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:51.012287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e9999-1067-44e2-97cc-3004cc2e0c67 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:51.014463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7246c38-d442-4d0e-95d2-1fea10c525ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:51.015215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7246c38-d442-4d0e-95d2-1fea10c525ac None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:51.015629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7246c38-d442-4d0e-95d2-1fea10c525ac None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:51.015979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7246c38-d442-4d0e-95d2-1fea10c525ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:51.098079 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7246c38-d442-4d0e-95d2-1fea10c525ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:51.099788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7246c38-d442-4d0e-95d2-1fea10c525ac None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:51.110381 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e9999-1067-44e2-97cc-3004cc2e0c67 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:51.120772 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:51.146866 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb149b-08ed-482a-91f1-d3c1992eb4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:51.147710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb149b-08ed-482a-91f1-d3c1992eb4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:51.148001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb149b-08ed-482a-91f1-d3c1992eb4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:51.148279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb149b-08ed-482a-91f1-d3c1992eb4d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:51.158990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb149b-08ed-482a-91f1-d3c1992eb4d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:51.160539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb149b-08ed-482a-91f1-d3c1992eb4d2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:51.177844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d6fb149b-08ed-482a-91f1-d3c1992eb4d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:51.177844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:51.177844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:51.177844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:51.177844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:51.177844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:51.177844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:51.177844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:51.177844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:51.177844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:51.177844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:51.177844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:51.177844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:51.177844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:51.177844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:51.178743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:51.179722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:51.179722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:51.179722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:51.179722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:51.179722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 320/650] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:51.180007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb149b-08ed-482a-91f1-d3c1992eb4d2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:51.208176 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 331/651] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:51.208176 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7246c38-d442-4d0e-95d2-1fea10c525ac None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:51.241912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:51.292448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb149b-08ed-482a-91f1-d3c1992eb4d2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:51.316194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7246c38-d442-4d0e-95d2-1fea10c525ac None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:51.328067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321103e-0b54-4d8d-8164-296a14c9aa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:51.328830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321103e-0b54-4d8d-8164-296a14c9aa01 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:51.329186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321103e-0b54-4d8d-8164-296a14c9aa01 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:51.329598 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321103e-0b54-4d8d-8164-296a14c9aa01 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:51.330391 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321103e-0b54-4d8d-8164-296a14c9aa01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:51.330916 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321103e-0b54-4d8d-8164-296a14c9aa01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:51.332938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321103e-0b54-4d8d-8164-296a14c9aa01 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:51.335509 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:51.351233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 321/652] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:51.351233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321103e-0b54-4d8d-8164-296a14c9aa01 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:51.357453 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adf3d6-eaa0-473f-9844-318959248093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:51.358232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adf3d6-eaa0-473f-9844-318959248093 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:51.358612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adf3d6-eaa0-473f-9844-318959248093 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:51.358984 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adf3d6-eaa0-473f-9844-318959248093 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:51.359838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adf3d6-eaa0-473f-9844-318959248093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:51.360238 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adf3d6-eaa0-473f-9844-318959248093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:51.361681 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adf3d6-eaa0-473f-9844-318959248093 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:51.379121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 332/653] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:51.380614 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adf3d6-eaa0-473f-9844-318959248093 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:51.460190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321103e-0b54-4d8d-8164-296a14c9aa01 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:51.473816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:51.485281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adf3d6-eaa0-473f-9844-318959248093 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:51.496454 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2d517-9375-44a8-96d8-c9453c940775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:51.497449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2d517-9375-44a8-96d8-c9453c940775 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:51.497894 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2d517-9375-44a8-96d8-c9453c940775 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:51.498180 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2d517-9375-44a8-96d8-c9453c940775 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:51.498906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2d517-9375-44a8-96d8-c9453c940775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:51.499241 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2d517-9375-44a8-96d8-c9453c940775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:51.500821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2d517-9375-44a8-96d8-c9453c940775 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:51.517995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 322/654] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:51.517995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2d517-9375-44a8-96d8-c9453c940775 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:51.525619 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81319109-dd0e-4450-879b-71dfb34eb30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:51.526490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81319109-dd0e-4450-879b-71dfb34eb30a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:51.526847 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81319109-dd0e-4450-879b-71dfb34eb30a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:51.527250 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81319109-dd0e-4450-879b-71dfb34eb30a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:51.537330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81319109-dd0e-4450-879b-71dfb34eb30a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:51.538902 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81319109-dd0e-4450-879b-71dfb34eb30a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:51.557530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-81319109-dd0e-4450-879b-71dfb34eb30a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:51.557530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:51.557530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:51.557530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:51.557530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:51.557530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:51.557530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:51.557530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:51.557530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:51.557530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:51.557530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:51.557530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:51.557530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:51.557530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:51.557530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:51.558605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:51.559607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:51.559607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:51.559607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:51.559607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:51.559607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 333/655] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:51.559974 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81319109-dd0e-4450-879b-71dfb34eb30a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:51.593415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:51.626491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2d517-9375-44a8-96d8-c9453c940775 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:51.663543 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97924295-ddbb-4be3-845a-c5d27a2fef67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:51.664206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97924295-ddbb-4be3-845a-c5d27a2fef67 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:51.664492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97924295-ddbb-4be3-845a-c5d27a2fef67 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:51.664783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97924295-ddbb-4be3-845a-c5d27a2fef67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:51.665599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97924295-ddbb-4be3-845a-c5d27a2fef67 None admin] RBAC: Adding query filter params (role.id=716cc96fb28146b7b9ae6271569a206f, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=51de59caa3144a1cbf12b20b15e2134f) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:51.665918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97924295-ddbb-4be3-845a-c5d27a2fef67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:51.667411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81319109-dd0e-4450-879b-71dfb34eb30a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:51.667523 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97924295-ddbb-4be3-845a-c5d27a2fef67 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:51.673756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 323/656] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:51 2022] GET /identity/v3/role_assignments?user.id=51de59caa3144a1cbf12b20b15e2134f&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=716cc96fb28146b7b9ae6271569a206f => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:51.674534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97924295-ddbb-4be3-845a-c5d27a2fef67 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:51.716696 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948c2d8-6512-45f6-b0c6-9c1633c98327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:51.717576 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948c2d8-6512-45f6-b0c6-9c1633c98327 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:51.717911 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948c2d8-6512-45f6-b0c6-9c1633c98327 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:51.718199 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948c2d8-6512-45f6-b0c6-9c1633c98327 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:51.724576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:51.727870 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948c2d8-6512-45f6-b0c6-9c1633c98327 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:51.729363 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948c2d8-6512-45f6-b0c6-9c1633c98327 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-1948c2d8-6512-45f6-b0c6-9c1633c98327 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:51.745669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:51.746704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:51.748532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:51.748532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:51.748532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:51.748532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:51.748532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:51.748532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:51.748532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:51.748532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:51.748532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 15:00:51.748532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:51.748532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 334/657] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:51.748532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948c2d8-6512-45f6-b0c6-9c1633c98327 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:51.811253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97924295-ddbb-4be3-845a-c5d27a2fef67 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:51.853838 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f75ed5-eb01-4b6c-ad82-bb9504a55a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:51.855038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f75ed5-eb01-4b6c-ad82-bb9504a55a53 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:51.855595 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f75ed5-eb01-4b6c-ad82-bb9504a55a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:51.856028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f75ed5-eb01-4b6c-ad82-bb9504a55a53 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:51.865702 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f75ed5-eb01-4b6c-ad82-bb9504a55a53 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:51.867138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f75ed5-eb01-4b6c-ad82-bb9504a55a53 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:51.876821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a2f75ed5-eb01-4b6c-ad82-bb9504a55a53 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:51.885017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:51.886022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:51.886976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:51.886976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:51.886976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:51.886976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:51.886976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:51.886976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:51.886976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:51.886976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:51.886976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 15:00:51.886976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:51.886976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 324/658] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:51.891053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f75ed5-eb01-4b6c-ad82-bb9504a55a53 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:51.894168 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948c2d8-6512-45f6-b0c6-9c1633c98327 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:51.941458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dac63d-8f60-4e50-9771-cffe092fd081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:51.942955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dac63d-8f60-4e50-9771-cffe092fd081 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:51.943362 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dac63d-8f60-4e50-9771-cffe092fd081 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:51.943653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dac63d-8f60-4e50-9771-cffe092fd081 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:51.944684 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dac63d-8f60-4e50-9771-cffe092fd081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:51.944791 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dac63d-8f60-4e50-9771-cffe092fd081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:51.946156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dac63d-8f60-4e50-9771-cffe092fd081 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:51.963240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 335/659] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:51.963240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dac63d-8f60-4e50-9771-cffe092fd081 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:52.008658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:52.017646 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f75ed5-eb01-4b6c-ad82-bb9504a55a53 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:52.054859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097f95f-6118-4444-88e3-029c58123c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:52.055973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097f95f-6118-4444-88e3-029c58123c64 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:52.056466 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097f95f-6118-4444-88e3-029c58123c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:52.056951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097f95f-6118-4444-88e3-029c58123c64 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:52.058008 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097f95f-6118-4444-88e3-029c58123c64 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:52.058521 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097f95f-6118-4444-88e3-029c58123c64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:52.060210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097f95f-6118-4444-88e3-029c58123c64 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:52.081460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 325/660] 10.222.0.67 () {56 vars in 1137 bytes} [Fri Jun 24 15:00:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:52.082927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097f95f-6118-4444-88e3-029c58123c64 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:52.094210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dac63d-8f60-4e50-9771-cffe092fd081 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:52.134473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369bff7-4194-4fde-9f9f-0dba9bca438f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:52.135641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369bff7-4194-4fde-9f9f-0dba9bca438f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:52.136135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369bff7-4194-4fde-9f9f-0dba9bca438f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:52.136772 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369bff7-4194-4fde-9f9f-0dba9bca438f None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:52.137875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8369bff7-4194-4fde-9f9f-0dba9bca438f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:52.138462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8369bff7-4194-4fde-9f9f-0dba9bca438f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:52.140176 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8369bff7-4194-4fde-9f9f-0dba9bca438f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:52.161802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 336/661] 10.222.0.67 () {56 vars in 1139 bytes} [Fri Jun 24 15:00:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:52.162948 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369bff7-4194-4fde-9f9f-0dba9bca438f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:52.175681 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:52.209185 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097f95f-6118-4444-88e3-029c58123c64 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:52.256229 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c49a07-5385-4792-a800-06c47f7d6d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:52.257042 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c49a07-5385-4792-a800-06c47f7d6d66 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:52.257323 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c49a07-5385-4792-a800-06c47f7d6d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:52.257620 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c49a07-5385-4792-a800-06c47f7d6d66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:52.266577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c49a07-5385-4792-a800-06c47f7d6d66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:52.268747 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c49a07-5385-4792-a800-06c47f7d6d66 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:52.284045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369bff7-4194-4fde-9f9f-0dba9bca438f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:52.286955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-96c49a07-5385-4792-a800-06c47f7d6d66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:52.286955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:52.286955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:52.286955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:52.286955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:52.286955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:52.286955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:52.286955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:52.286955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:52.286955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:52.286955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:52.286955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:52.286955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:52.286955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:52.286955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:52.287966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:52.288904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:52.288904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:52.288904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:52.288904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:52.288904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:52.288904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:52.288904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:52.288904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:52.291477 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 326/662] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:52.292496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c49a07-5385-4792-a800-06c47f7d6d66 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:52.313659 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:52.335064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdcc7b-db0e-443f-8d13-ec6953e130b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:52.335676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdcc7b-db0e-443f-8d13-ec6953e130b3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:52.335951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdcc7b-db0e-443f-8d13-ec6953e130b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:52.336226 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdcc7b-db0e-443f-8d13-ec6953e130b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:52.413899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c49a07-5385-4792-a800-06c47f7d6d66 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:52.415287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fdcc7b-db0e-443f-8d13-ec6953e130b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:52.416781 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fdcc7b-db0e-443f-8d13-ec6953e130b3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:52.427978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:52.454005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d822c26-7083-4963-8eda-39e7d9f464a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:52.454738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d822c26-7083-4963-8eda-39e7d9f464a1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:52.455025 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d822c26-7083-4963-8eda-39e7d9f464a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:52.455332 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d822c26-7083-4963-8eda-39e7d9f464a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:52.534072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d822c26-7083-4963-8eda-39e7d9f464a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:52.539975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 337/663] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:52.540147 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d822c26-7083-4963-8eda-39e7d9f464a1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:52.540254 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdcc7b-db0e-443f-8d13-ec6953e130b3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:52.555291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:52.642070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdcc7b-db0e-443f-8d13-ec6953e130b3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:52.647255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 327/664] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:52.647255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5058297-6f8a-42cc-9354-14b84fcad157 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:52.647255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5058297-6f8a-42cc-9354-14b84fcad157 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:52.647255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5058297-6f8a-42cc-9354-14b84fcad157 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:52.647255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 328/665] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:52.648017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5058297-6f8a-42cc-9354-14b84fcad157 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:52.659460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:52.683582 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad832a7-bff2-4343-9bc6-38742aa2ac66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:52.684250 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad832a7-bff2-4343-9bc6-38742aa2ac66 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:52.684533 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad832a7-bff2-4343-9bc6-38742aa2ac66 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:52.684809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad832a7-bff2-4343-9bc6-38742aa2ac66 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:52.685589 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad832a7-bff2-4343-9bc6-38742aa2ac66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:52.685897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad832a7-bff2-4343-9bc6-38742aa2ac66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:52.687354 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad832a7-bff2-4343-9bc6-38742aa2ac66 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:52.703646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 338/666] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:52.703646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad832a7-bff2-4343-9bc6-38742aa2ac66 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:52.752928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5058297-6f8a-42cc-9354-14b84fcad157 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:52.791474 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd4be1-a68e-42aa-b2e9-b9a2a521cffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:52.792320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd4be1-a68e-42aa-b2e9-b9a2a521cffb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:52.792600 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd4be1-a68e-42aa-b2e9-b9a2a521cffb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:52.792873 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd4be1-a68e-42aa-b2e9-b9a2a521cffb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:52.804392 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd4be1-a68e-42aa-b2e9-b9a2a521cffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:52.804708 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad832a7-bff2-4343-9bc6-38742aa2ac66 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:52.806054 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd4be1-a68e-42aa-b2e9-b9a2a521cffb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:52.824163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-befd4be1-a68e-42aa-b2e9-b9a2a521cffb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:52.824163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:52.824163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:52.824163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:52.824163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:52.824163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:52.824163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:52.824163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:52.824163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:52.824163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:52.824163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:52.824163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:52.824163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:52.824163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:52.824163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:52.825805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:52.826983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:52.826983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:52.826983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:52.826983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:52.831276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 329/667] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:52.832612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930302e0-f3ba-4253-a2d4-cc1ee1baa772 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:52.832949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930302e0-f3ba-4253-a2d4-cc1ee1baa772 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:52.833236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930302e0-f3ba-4253-a2d4-cc1ee1baa772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:52.842000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64169d47-94f0-42bf-8ae9-30c4b6edbf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:52.843020 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64169d47-94f0-42bf-8ae9-30c4b6edbf71 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:52.843562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64169d47-94f0-42bf-8ae9-30c4b6edbf71 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:52.849245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64169d47-94f0-42bf-8ae9-30c4b6edbf71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:52.849371 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:52.853210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-930302e0-f3ba-4253-a2d4-cc1ee1baa772 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:52.854481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-930302e0-f3ba-4253-a2d4-cc1ee1baa772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:52.854991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64169d47-94f0-42bf-8ae9-30c4b6edbf71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:52.856743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64169d47-94f0-42bf-8ae9-30c4b6edbf71 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:52.876808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-64169d47-94f0-42bf-8ae9-30c4b6edbf71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:52.876808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:52.876808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:52.876808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:52.876808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:52.876808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:52.876808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:52.876808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:52.876808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:52.876808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:52.876808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:52.876808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:52.876808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:52.876808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:52.876808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:52.878335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:52.880168 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:52.880168 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:52.880168 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:52.880168 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:52.886612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 339/668] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:52.887513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64169d47-94f0-42bf-8ae9-30c4b6edbf71 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:52.910488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:52.933012 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 330/669] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:52.934263 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930302e0-f3ba-4253-a2d4-cc1ee1baa772 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:53.007008 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64169d47-94f0-42bf-8ae9-30c4b6edbf71 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:53.042381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120f32a-4109-4daf-b598-ab97a6393724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:53.043040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120f32a-4109-4daf-b598-ab97a6393724 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:53.043361 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120f32a-4109-4daf-b598-ab97a6393724 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:53.043636 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120f32a-4109-4daf-b598-ab97a6393724 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:53.044472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4120f32a-4109-4daf-b598-ab97a6393724 None admin] RBAC: Adding query filter params (role.id=716cc96fb28146b7b9ae6271569a206f, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=1971c464293f426f97d6cb199e48193b) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:53.044770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4120f32a-4109-4daf-b598-ab97a6393724 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:53.046190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4120f32a-4109-4daf-b598-ab97a6393724 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:53.050479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:53.053191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 340/670] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:52 2022] GET /identity/v3/role_assignments?user.id=1971c464293f426f97d6cb199e48193b&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=716cc96fb28146b7b9ae6271569a206f => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:53.053890 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120f32a-4109-4daf-b598-ab97a6393724 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:53.070235 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930302e0-f3ba-4253-a2d4-cc1ee1baa772 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:53.124090 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790aee1-b8b4-4df8-b6ae-09c1e17e7746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:53.125058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790aee1-b8b4-4df8-b6ae-09c1e17e7746 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:53.125494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790aee1-b8b4-4df8-b6ae-09c1e17e7746 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:53.125965 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790aee1-b8b4-4df8-b6ae-09c1e17e7746 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:53.126860 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790aee1-b8b4-4df8-b6ae-09c1e17e7746 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:53.127365 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790aee1-b8b4-4df8-b6ae-09c1e17e7746 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:53.128910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790aee1-b8b4-4df8-b6ae-09c1e17e7746 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:53.142053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 331/671] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:53.143273 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790aee1-b8b4-4df8-b6ae-09c1e17e7746 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:53.171486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:53.198611 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120f32a-4109-4daf-b598-ab97a6393724 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:53.249532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc6a4c-ba47-48b0-b7da-a7ffc2c39ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:53.250537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc6a4c-ba47-48b0-b7da-a7ffc2c39ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:53.250896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc6a4c-ba47-48b0-b7da-a7ffc2c39ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:53.251274 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc6a4c-ba47-48b0-b7da-a7ffc2c39ee2 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:53.252273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc6a4c-ba47-48b0-b7da-a7ffc2c39ee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:53.252691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc6a4c-ba47-48b0-b7da-a7ffc2c39ee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:53.254906 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc6a4c-ba47-48b0-b7da-a7ffc2c39ee2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:53.262004 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790aee1-b8b4-4df8-b6ae-09c1e17e7746 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:53.272091 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 341/672] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:53.273422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc6a4c-ba47-48b0-b7da-a7ffc2c39ee2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:53.305236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1ee7e-1961-4825-a7bf-651fd1db329a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:53.305702 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1ee7e-1961-4825-a7bf-651fd1db329a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:53.307323 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1ee7e-1961-4825-a7bf-651fd1db329a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:53.307820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1ee7e-1961-4825-a7bf-651fd1db329a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:53.319726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1ee7e-1961-4825-a7bf-651fd1db329a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d9d1ee7e-1961-4825-a7bf-651fd1db329a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:53.327438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:53.328689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:53.329692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:53.329692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:53.329692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:53.329692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:53.329692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:53.329692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:53.329692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:53.329692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:53.329692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:53.329692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:53.329692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:53.329692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:53.329692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 332/673] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:53.330429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1ee7e-1961-4825-a7bf-651fd1db329a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:53.350963 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:53.407444 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc6a4c-ba47-48b0-b7da-a7ffc2c39ee2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:53.446670 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1ee7e-1961-4825-a7bf-651fd1db329a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:53.459665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7498cc-a21f-4071-b6a3-aa70c2a40673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:53.460990 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7498cc-a21f-4071-b6a3-aa70c2a40673 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:53.461465 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7498cc-a21f-4071-b6a3-aa70c2a40673 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:53.461901 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7498cc-a21f-4071-b6a3-aa70c2a40673 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:53.475291 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7498cc-a21f-4071-b6a3-aa70c2a40673 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:53.476917 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7498cc-a21f-4071-b6a3-aa70c2a40673 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:53.491268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1232f-358b-434a-8dad-cad7031dadbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:53.492307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1232f-358b-434a-8dad-cad7031dadbd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:53.492748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1232f-358b-434a-8dad-cad7031dadbd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:53.493170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1232f-358b-434a-8dad-cad7031dadbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:53.493819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4c7498cc-a21f-4071-b6a3-aa70c2a40673 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:53.493819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:53.493819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:53.493819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:53.493819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:53.493819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:53.493819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:53.493819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:53.493819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:53.493819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:53.493819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:53.493819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:53.493819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:53.493819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:53.493819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:53.494881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:53.496040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:53.496040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:53.496040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:53.496040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:53.496040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:53.496040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:53.496040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:53.496040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:53.502540 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 342/674] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:53.502540 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7498cc-a21f-4071-b6a3-aa70c2a40673 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:53.505167 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:53.511764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1232f-358b-434a-8dad-cad7031dadbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:53.513826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1232f-358b-434a-8dad-cad7031dadbd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:53.533665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-cfb1232f-358b-434a-8dad-cad7031dadbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:53.533665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:53.533665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:53.533665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:53.533665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:53.533665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:53.533665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:53.533665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:53.533665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:53.533665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:53.533665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:53.533665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:53.533665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:53.533665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:53.533665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:53.534807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:53.536467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:53.536467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:53.536467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:53.536467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:53.536467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:53.536467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:53.536467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:53.536467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:53.539319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 333/675] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:53.540450 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1232f-358b-434a-8dad-cad7031dadbd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:53.622953 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7498cc-a21f-4071-b6a3-aa70c2a40673 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:53.632727 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:53.657839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1232f-358b-434a-8dad-cad7031dadbd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:53.669215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1f437-dd21-4c0a-8beb-da9f94353571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:53.670164 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1f437-dd21-4c0a-8beb-da9f94353571 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:53.671058 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1f437-dd21-4c0a-8beb-da9f94353571 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:53.671509 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1f437-dd21-4c0a-8beb-da9f94353571 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:53.672833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1f437-dd21-4c0a-8beb-da9f94353571 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:53.673599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1f437-dd21-4c0a-8beb-da9f94353571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:53.675443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1f437-dd21-4c0a-8beb-da9f94353571 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:53.685857 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 343/676] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:53.686906 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1f437-dd21-4c0a-8beb-da9f94353571 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:53.715475 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bda8bc-937b-4aa9-a338-9725be2662f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:53.716087 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bda8bc-937b-4aa9-a338-9725be2662f2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:53.716609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bda8bc-937b-4aa9-a338-9725be2662f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:53.716891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bda8bc-937b-4aa9-a338-9725be2662f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:53.718099 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bda8bc-937b-4aa9-a338-9725be2662f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:53.718415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bda8bc-937b-4aa9-a338-9725be2662f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:53.720108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bda8bc-937b-4aa9-a338-9725be2662f2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:53.741683 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 334/677] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:53.741683 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bda8bc-937b-4aa9-a338-9725be2662f2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:53.780333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:53.819369 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1f437-dd21-4c0a-8beb-da9f94353571 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:53.859255 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63680b1c-ffb8-4084-ab8c-125fae58bc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:53.860312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63680b1c-ffb8-4084-ab8c-125fae58bc4d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:53.860604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63680b1c-ffb8-4084-ab8c-125fae58bc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:53.860887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63680b1c-ffb8-4084-ab8c-125fae58bc4d None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:53.861684 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63680b1c-ffb8-4084-ab8c-125fae58bc4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:53.861999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63680b1c-ffb8-4084-ab8c-125fae58bc4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:53.863977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63680b1c-ffb8-4084-ab8c-125fae58bc4d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:53.877608 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bda8bc-937b-4aa9-a338-9725be2662f2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:53.887262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 344/678] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:53.888788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63680b1c-ffb8-4084-ab8c-125fae58bc4d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:53.919572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c29764-25fc-402d-aaab-e8fb111715b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:53.920369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c29764-25fc-402d-aaab-e8fb111715b4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:53.920667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c29764-25fc-402d-aaab-e8fb111715b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:53.921329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c29764-25fc-402d-aaab-e8fb111715b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:53.924270 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:53.933102 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c29764-25fc-402d-aaab-e8fb111715b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:53.934758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c29764-25fc-402d-aaab-e8fb111715b4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:53.953064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-58c29764-25fc-402d-aaab-e8fb111715b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:53.953064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:53.953064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:53.953064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:53.953064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:53.953064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:53.953064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:53.953064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:53.953064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:53.953064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:53.953064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:53.953064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:53.953064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:53.953064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:53.953064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:53.954158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:53.955212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:53.955212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:53.955212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:53.955212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:53.957706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 335/679] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:53.959264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c29764-25fc-402d-aaab-e8fb111715b4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:54.007995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63680b1c-ffb8-4084-ab8c-125fae58bc4d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:54.034110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:54.043831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37476e20-9784-44d4-96f7-aa22cdcb9746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:54.060812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37476e20-9784-44d4-96f7-aa22cdcb9746 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:54.060812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37476e20-9784-44d4-96f7-aa22cdcb9746 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:54.060812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37476e20-9784-44d4-96f7-aa22cdcb9746 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:54.060812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37476e20-9784-44d4-96f7-aa22cdcb9746 None admin] RBAC: Adding query filter params (role.id=716cc96fb28146b7b9ae6271569a206f, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=2efb120c084a48858d32f97843d76834) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:54.060812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37476e20-9784-44d4-96f7-aa22cdcb9746 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:54.060812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37476e20-9784-44d4-96f7-aa22cdcb9746 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:54.061498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 345/680] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:53 2022] GET /identity/v3/role_assignments?user.id=2efb120c084a48858d32f97843d76834&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=716cc96fb28146b7b9ae6271569a206f => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:54.061498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b8b43-e431-40ba-8d82-9db42b2205af None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:54.061498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b8b43-e431-40ba-8d82-9db42b2205af None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:54.061498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b8b43-e431-40ba-8d82-9db42b2205af None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:54.061498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 346/681] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:54.061498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b8b43-e431-40ba-8d82-9db42b2205af None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:54.082861 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c29764-25fc-402d-aaab-e8fb111715b4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:54.128842 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91336c-677b-4c68-b58d-407d9cee3c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:54.130150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91336c-677b-4c68-b58d-407d9cee3c94 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:54.130427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91336c-677b-4c68-b58d-407d9cee3c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:54.130705 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91336c-677b-4c68-b58d-407d9cee3c94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:54.131577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91336c-677b-4c68-b58d-407d9cee3c94 None admin] RBAC: Adding query filter params (role.id=716cc96fb28146b7b9ae6271569a206f, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=95e7fb8f48dd4348b54e3694a8410bfd) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:54.132454 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91336c-677b-4c68-b58d-407d9cee3c94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:54.134253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91336c-677b-4c68-b58d-407d9cee3c94 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:54.147364 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 336/682] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:53 2022] GET /identity/v3/role_assignments?user.id=95e7fb8f48dd4348b54e3694a8410bfd&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=716cc96fb28146b7b9ae6271569a206f => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:54.147364 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3294f4-5575-4979-8c3c-c8c11e6594b7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:54.147364 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3294f4-5575-4979-8c3c-c8c11e6594b7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:54.147364 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3294f4-5575-4979-8c3c-c8c11e6594b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:54.167552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ef3294f4-5575-4979-8c3c-c8c11e6594b7 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:54.169111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ef3294f4-5575-4979-8c3c-c8c11e6594b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:54.232820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:54.259617 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b8b43-e431-40ba-8d82-9db42b2205af None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:54.260887 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 337/683] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:54.287464 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3294f4-5575-4979-8c3c-c8c11e6594b7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:54.301922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86cdc5-1bcf-4002-9148-42267c4b8ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:54.301922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86cdc5-1bcf-4002-9148-42267c4b8ddb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:54.302699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86cdc5-1bcf-4002-9148-42267c4b8ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:54.303092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86cdc5-1bcf-4002-9148-42267c4b8ddb None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:54.303984 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86cdc5-1bcf-4002-9148-42267c4b8ddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:54.304662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86cdc5-1bcf-4002-9148-42267c4b8ddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:54.305957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86cdc5-1bcf-4002-9148-42267c4b8ddb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:54.320250 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 347/684] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:54.328311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86cdc5-1bcf-4002-9148-42267c4b8ddb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:54.387725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:54.417040 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3294f4-5575-4979-8c3c-c8c11e6594b7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:54.457742 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86cdc5-1bcf-4002-9148-42267c4b8ddb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:54.470283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899215ad-3e86-47e8-a1bb-f2c5061fb175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:54.471683 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899215ad-3e86-47e8-a1bb-f2c5061fb175 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:54.472241 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899215ad-3e86-47e8-a1bb-f2c5061fb175 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:54.472721 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899215ad-3e86-47e8-a1bb-f2c5061fb175 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:54.487254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899215ad-3e86-47e8-a1bb-f2c5061fb175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:54.497977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d98da1-bfb3-4ee1-b9a3-b835a35e69e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-899215ad-3e86-47e8-a1bb-f2c5061fb175 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:54.498299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:54.499438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:54.500467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:54.500467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:54.500467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:54.500467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:54.500467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:54.500467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:54.500467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:54.500467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:54.500467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:54.500467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:54.500467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:00:54.500467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:54.501635 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d98da1-bfb3-4ee1-b9a3-b835a35e69e1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:54.502017 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d98da1-bfb3-4ee1-b9a3-b835a35e69e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:54.502391 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d98da1-bfb3-4ee1-b9a3-b835a35e69e1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:54.503439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 338/685] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:00:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:54.509560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899215ad-3e86-47e8-a1bb-f2c5061fb175 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:54.512376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d98da1-bfb3-4ee1-b9a3-b835a35e69e1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:54.514914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d98da1-bfb3-4ee1-b9a3-b835a35e69e1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-56d98da1-bfb3-4ee1-b9a3-b835a35e69e1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:54.533056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:54.534287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:54.535646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:54.535646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:54.535646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:54.535646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:54.535646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:54.535646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:54.535646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:54.535646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:54.535646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 15:00:54.535646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:54.537399 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 348/686] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:00:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:54.543034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d98da1-bfb3-4ee1-b9a3-b835a35e69e1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:54.573346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:54.649290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899215ad-3e86-47e8-a1bb-f2c5061fb175 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:54.672969 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d98da1-bfb3-4ee1-b9a3-b835a35e69e1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:54.688210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:54.694214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8dc278-d9ba-4a3a-b2f2-b9f9bd9c38d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:54.695104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8dc278-d9ba-4a3a-b2f2-b9f9bd9c38d3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:54.695570 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8dc278-d9ba-4a3a-b2f2-b9f9bd9c38d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:54.696130 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8dc278-d9ba-4a3a-b2f2-b9f9bd9c38d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:54.697417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8dc278-d9ba-4a3a-b2f2-b9f9bd9c38d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:54.697883 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8dc278-d9ba-4a3a-b2f2-b9f9bd9c38d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:54.699453 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8dc278-d9ba-4a3a-b2f2-b9f9bd9c38d3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:54.711796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 339/687] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:54.714946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1886aa-db2b-4799-8f6e-5cac84690a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:54.720782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1886aa-db2b-4799-8f6e-5cac84690a78 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:54.720782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1886aa-db2b-4799-8f6e-5cac84690a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:54.720782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1886aa-db2b-4799-8f6e-5cac84690a78 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:54.720782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1886aa-db2b-4799-8f6e-5cac84690a78 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:54.720782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1886aa-db2b-4799-8f6e-5cac84690a78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:54.720782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1886aa-db2b-4799-8f6e-5cac84690a78 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:54.724033 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8dc278-d9ba-4a3a-b2f2-b9f9bd9c38d3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:54.743131 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 349/688] 10.222.0.67 () {56 vars in 1135 bytes} [Fri Jun 24 15:00:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:54.752200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1886aa-db2b-4799-8f6e-5cac84690a78 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:54.882178 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8dc278-d9ba-4a3a-b2f2-b9f9bd9c38d3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:54.884574 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:54.922962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8354b2-b390-4895-8f29-b6b52542dc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:54.923806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8354b2-b390-4895-8f29-b6b52542dc0e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:54.924086 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8354b2-b390-4895-8f29-b6b52542dc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:54.924360 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8354b2-b390-4895-8f29-b6b52542dc0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:54.945991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8354b2-b390-4895-8f29-b6b52542dc0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:54.948347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8354b2-b390-4895-8f29-b6b52542dc0e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:54.954808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1886aa-db2b-4799-8f6e-5cac84690a78 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-aa8354b2-b390-4895-8f29-b6b52542dc0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:54.981806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:54.982947 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:54.982947 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:54.982947 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:54.982947 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:54.982947 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 340/689] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:54.987752 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8354b2-b390-4895-8f29-b6b52542dc0e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:55.029450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddde33-d28c-4693-9b9f-eb2ff1b2acd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:55.030457 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddde33-d28c-4693-9b9f-eb2ff1b2acd1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:55.030928 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddde33-d28c-4693-9b9f-eb2ff1b2acd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:55.031418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddde33-d28c-4693-9b9f-eb2ff1b2acd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:55.110853 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8354b2-b390-4895-8f29-b6b52542dc0e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:55.114637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:55.156851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a13a22-330b-48f3-be1c-41c6bbb74159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:55.157889 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a13a22-330b-48f3-be1c-41c6bbb74159 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:55.158436 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a13a22-330b-48f3-be1c-41c6bbb74159 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:55.158822 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a13a22-330b-48f3-be1c-41c6bbb74159 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:55.159741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a13a22-330b-48f3-be1c-41c6bbb74159 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:55.160732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a13a22-330b-48f3-be1c-41c6bbb74159 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:55.162276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a13a22-330b-48f3-be1c-41c6bbb74159 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:55.172471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddde33-d28c-4693-9b9f-eb2ff1b2acd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:55.174258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddde33-d28c-4693-9b9f-eb2ff1b2acd1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:55.187815 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 341/690] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:55.192955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a13a22-330b-48f3-be1c-41c6bbb74159 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:55.224733 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:55.287813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 350/691] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:55.292885 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddde33-d28c-4693-9b9f-eb2ff1b2acd1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:55.343459 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a13a22-330b-48f3-be1c-41c6bbb74159 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:55.380973 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:55.404087 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb79d1d-e9c3-4bce-a1c5-9bbff3a2582f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:55.405285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb79d1d-e9c3-4bce-a1c5-9bbff3a2582f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:55.405788 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb79d1d-e9c3-4bce-a1c5-9bbff3a2582f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:55.406296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb79d1d-e9c3-4bce-a1c5-9bbff3a2582f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:55.416976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb79d1d-e9c3-4bce-a1c5-9bbff3a2582f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:55.419104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb79d1d-e9c3-4bce-a1c5-9bbff3a2582f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-cbb79d1d-e9c3-4bce-a1c5-9bbff3a2582f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:00:55.436442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:55.437825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:55.438884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:55.438884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:55.438884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:00:55.438884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:00:55.438884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:00:55.438884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:00:55.438884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:00:55.438884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:00:55.438884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:00:55.438884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:55.443995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 342/692] 10.222.0.67 () {56 vars in 1093 bytes} [Fri Jun 24 15:00:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:55.449506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb79d1d-e9c3-4bce-a1c5-9bbff3a2582f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:55.456451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddde33-d28c-4693-9b9f-eb2ff1b2acd1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:55.516474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9926d-0b9c-4c7d-a612-fedcb2331dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:55.517526 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9926d-0b9c-4c7d-a612-fedcb2331dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:55.517808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9926d-0b9c-4c7d-a612-fedcb2331dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:55.518080 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9926d-0b9c-4c7d-a612-fedcb2331dd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:55.534291 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9926d-0b9c-4c7d-a612-fedcb2331dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:55.537614 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9926d-0b9c-4c7d-a612-fedcb2331dd1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:55.544023 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:55.560674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9df9926d-0b9c-4c7d-a612-fedcb2331dd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:55.560674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:55.560674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:55.560674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:55.560674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:55.560674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:55.560674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:55.560674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:55.560674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:55.560674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:55.560674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:55.560674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:55.560674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:55.560674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:55.560674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:55.562317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:55.563609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:55.563609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:55.563609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:55.563609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:55.567123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 351/693] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:00:55.568797 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c2e75-f1ab-4b7d-b5e6-e936a8e93cc8 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:55.569084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c2e75-f1ab-4b7d-b5e6-e936a8e93cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:55.569372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c2e75-f1ab-4b7d-b5e6-e936a8e93cc8 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:55.575303 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 352/694] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:55.576344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c2e75-f1ab-4b7d-b5e6-e936a8e93cc8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:55.578393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb79d1d-e9c3-4bce-a1c5-9bbff3a2582f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:55.614936 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db10477-3f69-40b9-bcb4-25c21d3e4ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:55.615990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db10477-3f69-40b9-bcb4-25c21d3e4ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:55.616473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db10477-3f69-40b9-bcb4-25c21d3e4ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:55.616926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db10477-3f69-40b9-bcb4-25c21d3e4ee2 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:55.618217 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db10477-3f69-40b9-bcb4-25c21d3e4ee2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:55.618765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db10477-3f69-40b9-bcb4-25c21d3e4ee2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:55.620528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db10477-3f69-40b9-bcb4-25c21d3e4ee2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:55.639255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 343/695] 10.222.0.67 () {56 vars in 1133 bytes} [Fri Jun 24 15:00:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:55.640937 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80309d01-f402-4015-9d16-499fae1718f8 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:55.641483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80309d01-f402-4015-9d16-499fae1718f8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:55.641951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80309d01-f402-4015-9d16-499fae1718f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:55.658809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-80309d01-f402-4015-9d16-499fae1718f8 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:55.660291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-80309d01-f402-4015-9d16-499fae1718f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:55.708186 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c2e75-f1ab-4b7d-b5e6-e936a8e93cc8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:55.732895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:55.745777 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 344/696] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:55.747600 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80309d01-f402-4015-9d16-499fae1718f8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:55.762440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079529db-f49e-4d93-a88d-77b0670eda85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:55.763444 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079529db-f49e-4d93-a88d-77b0670eda85 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:55.764031 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079529db-f49e-4d93-a88d-77b0670eda85 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:55.764537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079529db-f49e-4d93-a88d-77b0670eda85 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:55.766088 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079529db-f49e-4d93-a88d-77b0670eda85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:55.767783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079529db-f49e-4d93-a88d-77b0670eda85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:55.769469 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079529db-f49e-4d93-a88d-77b0670eda85 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:55.785707 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 353/697] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:55.786710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079529db-f49e-4d93-a88d-77b0670eda85 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:55.825726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:55.865633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80309d01-f402-4015-9d16-499fae1718f8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:55.893663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079529db-f49e-4d93-a88d-77b0670eda85 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:55.904246 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0582d-35b6-45ec-b33a-2b11a34457c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:55.905543 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0582d-35b6-45ec-b33a-2b11a34457c1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:55.906006 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0582d-35b6-45ec-b33a-2b11a34457c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:55.906493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0582d-35b6-45ec-b33a-2b11a34457c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:55.931272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e8716-7b04-429c-b140-019d241154a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:55.932076 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e8716-7b04-429c-b140-019d241154a2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:55.932360 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e8716-7b04-429c-b140-019d241154a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:55.932636 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e8716-7b04-429c-b140-019d241154a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:55.944955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e8716-7b04-429c-b140-019d241154a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:55.947221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e8716-7b04-429c-b140-019d241154a2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:55.976094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ad8e8716-7b04-429c-b140-019d241154a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:55.976094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:55.976094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:55.976094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:55.976094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:55.976094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:55.976094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:55.976094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:55.976094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:55.976094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:55.976094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:55.976094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:55.976094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:55.976094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:55.976094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:55.977458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:55.978343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:55.978343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:55.978343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:55.978343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:55.981450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 354/698] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:55.982476 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e8716-7b04-429c-b140-019d241154a2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:55.992489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0582d-35b6-45ec-b33a-2b11a34457c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:55.993956 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0582d-35b6-45ec-b33a-2b11a34457c1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:56.025030 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:56.073112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:56.119507 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 345/699] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:00:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 15:00:56.120281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e8716-7b04-429c-b140-019d241154a2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:56.120944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0582d-35b6-45ec-b33a-2b11a34457c1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:56.169744 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefa526-bddf-4f1d-b71a-e6a2b98bc612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:56.171033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefa526-bddf-4f1d-b71a-e6a2b98bc612 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:56.171410 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefa526-bddf-4f1d-b71a-e6a2b98bc612 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:56.172018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefa526-bddf-4f1d-b71a-e6a2b98bc612 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:56.181416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefa526-bddf-4f1d-b71a-e6a2b98bc612 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:56.183762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefa526-bddf-4f1d-b71a-e6a2b98bc612 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:56.202929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-6aefa526-bddf-4f1d-b71a-e6a2b98bc612 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:56.202929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:56.202929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:56.202929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:56.202929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:56.202929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:56.202929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:56.202929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:56.202929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:56.202929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:56.202929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:56.202929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:56.202929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:56.202929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:56.202929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:56.203888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:56.204788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:56.204788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:56.204788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:56.204788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:56.204788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:56.204788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:56.204788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:56.204788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:56.204788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 355/700] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:56.210134 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefa526-bddf-4f1d-b71a-e6a2b98bc612 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:56.237847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0582d-35b6-45ec-b33a-2b11a34457c1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:56.276731 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b4643-005e-4de1-bd09-00eba3ce5c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:56.277759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b4643-005e-4de1-bd09-00eba3ce5c62 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:56.278262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b4643-005e-4de1-bd09-00eba3ce5c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:56.278730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b4643-005e-4de1-bd09-00eba3ce5c62 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:56.279691 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b4643-005e-4de1-bd09-00eba3ce5c62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:56.280194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b4643-005e-4de1-bd09-00eba3ce5c62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:56.281790 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b4643-005e-4de1-bd09-00eba3ce5c62 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:56.298692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 346/701] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:56.299914 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b4643-005e-4de1-bd09-00eba3ce5c62 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:56.325484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:56.328236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:56.337780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefa526-bddf-4f1d-b71a-e6a2b98bc612 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:56.380657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4af3cf-30bb-4af0-9245-1633fd09dff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:56.381912 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4af3cf-30bb-4af0-9245-1633fd09dff3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:56.382393 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4af3cf-30bb-4af0-9245-1633fd09dff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:56.382887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4af3cf-30bb-4af0-9245-1633fd09dff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:56.395094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4af3cf-30bb-4af0-9245-1633fd09dff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:56.397104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4af3cf-30bb-4af0-9245-1633fd09dff3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:56.416883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-2a4af3cf-30bb-4af0-9245-1633fd09dff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:56.416883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:56.416883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:56.416883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:56.416883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:56.416883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:56.416883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:56.416883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:56.416883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:56.416883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:56.416883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:56.416883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:56.416883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:00:56.416883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:00:56.416883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:56.417837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:56.418743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:00:56.418743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:00:56.418743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:00:56.418743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:56.418977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b4643-005e-4de1-bd09-00eba3ce5c62 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:56.425441 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 356/702] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:56.427682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4af3cf-30bb-4af0-9245-1633fd09dff3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:56.458149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16620f42-08ef-4e13-b139-a3a62ce5f4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:56.459191 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16620f42-08ef-4e13-b139-a3a62ce5f4fc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:56.459541 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16620f42-08ef-4e13-b139-a3a62ce5f4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:56.459826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16620f42-08ef-4e13-b139-a3a62ce5f4fc None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:56.460636 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16620f42-08ef-4e13-b139-a3a62ce5f4fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:56.460946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16620f42-08ef-4e13-b139-a3a62ce5f4fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:56.462390 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16620f42-08ef-4e13-b139-a3a62ce5f4fc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:56.475613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 347/703] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:56.479579 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16620f42-08ef-4e13-b139-a3a62ce5f4fc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:56.542603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4af3cf-30bb-4af0-9245-1633fd09dff3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:56.566351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:56.577285 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d20b41-553b-4621-9b15-8ebe7fa729d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:56.577984 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d20b41-553b-4621-9b15-8ebe7fa729d1 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:56.578267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d20b41-553b-4621-9b15-8ebe7fa729d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:56.578658 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d20b41-553b-4621-9b15-8ebe7fa729d1 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:56.579258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d20b41-553b-4621-9b15-8ebe7fa729d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:56.580695 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d20b41-553b-4621-9b15-8ebe7fa729d1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:56.583017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16620f42-08ef-4e13-b139-a3a62ce5f4fc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:56.617780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd41c12-aa6b-4390-af0f-e2a47ffe35e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:56.618470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd41c12-aa6b-4390-af0f-e2a47ffe35e9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:56.618782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd41c12-aa6b-4390-af0f-e2a47ffe35e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:56.619031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd41c12-aa6b-4390-af0f-e2a47ffe35e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:56.619791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd41c12-aa6b-4390-af0f-e2a47ffe35e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:56.620098 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd41c12-aa6b-4390-af0f-e2a47ffe35e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:56.621476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd41c12-aa6b-4390-af0f-e2a47ffe35e9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:56.632660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 348/704] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:56.633967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49df30-4f1b-4fc4-a7c9-f0369cbcc625 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:56.634257 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49df30-4f1b-4fc4-a7c9-f0369cbcc625 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:56.634531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49df30-4f1b-4fc4-a7c9-f0369cbcc625 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:56.639234 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 349/705] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:56.639234 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49df30-4f1b-4fc4-a7c9-f0369cbcc625 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:56.656959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:56.664170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 357/706] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 15:00:56 2022] POST /identity/v3/users => generated 254 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:00:56.671410 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d20b41-553b-4621-9b15-8ebe7fa729d1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:56.777384 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49df30-4f1b-4fc4-a7c9-f0369cbcc625 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:56.791111 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d20b41-553b-4621-9b15-8ebe7fa729d1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:56.830351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8346d6-331b-4a87-8663-9c025374faba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:56.831109 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8346d6-331b-4a87-8663-9c025374faba None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:56.831439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8346d6-331b-4a87-8663-9c025374faba None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:56.831841 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8346d6-331b-4a87-8663-9c025374faba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:56.833350 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8346d6-331b-4a87-8663-9c025374faba None admin] RBAC: Adding query filter params (role.id=716cc96fb28146b7b9ae6271569a206f, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=e4732ab3c8c446958092246d16b4b4f0) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:56.833686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8346d6-331b-4a87-8663-9c025374faba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:56.836271 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d81ede-6e51-4747-9c8f-699152535b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:56.836404 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8346d6-331b-4a87-8663-9c025374faba None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:56.841637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d81ede-6e51-4747-9c8f-699152535b50 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:56.841637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d81ede-6e51-4747-9c8f-699152535b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:56.841637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d81ede-6e51-4747-9c8f-699152535b50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:56.844245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 350/707] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:56 2022] GET /identity/v3/role_assignments?user.id=e4732ab3c8c446958092246d16b4b4f0&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=716cc96fb28146b7b9ae6271569a206f => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:56.845903 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574b613-9a9e-4c4e-93c8-2d3291f6ea49 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:56.846195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574b613-9a9e-4c4e-93c8-2d3291f6ea49 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:56.846365 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d81ede-6e51-4747-9c8f-699152535b50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:56.846537 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574b613-9a9e-4c4e-93c8-2d3291f6ea49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:56.848787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d81ede-6e51-4747-9c8f-699152535b50 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:56.865022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e2d81ede-6e51-4747-9c8f-699152535b50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:56.865022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:56.865022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:56.865022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:56.865022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:56.865022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:56.865022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:56.865022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:56.865022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:56.865022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:56.865022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:56.865022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:56.865022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:00:56.865022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:00:56.865022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:56.871651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:56.872647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:00:56.872647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:00:56.872647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:00:56.872647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:00:56.872647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:00:56.872647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:00:56.872647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:00:56.872647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:56.873131 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2574b613-9a9e-4c4e-93c8-2d3291f6ea49 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:56.873221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 358/708] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:56.873301 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2574b613-9a9e-4c4e-93c8-2d3291f6ea49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:56.873301 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:56.873439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2c211-f654-4df7-af19-189c4dac4e9d None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:56.873439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2c211-f654-4df7-af19-189c4dac4e9d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:56.873439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2c211-f654-4df7-af19-189c4dac4e9d None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:56.874742 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 359/709] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:56.875760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2c211-f654-4df7-af19-189c4dac4e9d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:56.959039 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:56.978016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 351/710] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:56.980543 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72129354-482f-4622-992e-b2ecdf485665 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:56.980848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72129354-482f-4622-992e-b2ecdf485665 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:56.981584 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72129354-482f-4622-992e-b2ecdf485665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:57.005195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-72129354-482f-4622-992e-b2ecdf485665 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:57.006615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-72129354-482f-4622-992e-b2ecdf485665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:57.018391 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2c211-f654-4df7-af19-189c4dac4e9d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:57.059797 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319d89b-20e1-4e62-b64c-0030af61ff28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:57.061062 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319d89b-20e1-4e62-b64c-0030af61ff28 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:57.061815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319d89b-20e1-4e62-b64c-0030af61ff28 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:57.062104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319d89b-20e1-4e62-b64c-0030af61ff28 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:57.062913 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319d89b-20e1-4e62-b64c-0030af61ff28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:57.063537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319d89b-20e1-4e62-b64c-0030af61ff28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:57.065447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319d89b-20e1-4e62-b64c-0030af61ff28 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:57.081327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 360/711] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:00:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:57.082767 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319d89b-20e1-4e62-b64c-0030af61ff28 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:57.098092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 352/712] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:57.099138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72129354-482f-4622-992e-b2ecdf485665 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:57.172988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:57.214822 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72129354-482f-4622-992e-b2ecdf485665 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:57.217508 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319d89b-20e1-4e62-b64c-0030af61ff28 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:57.241041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:57.250929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8821c8-4c7d-4792-87ab-d7fc2487bee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:57.251743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8821c8-4c7d-4792-87ab-d7fc2487bee6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:57.252079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8821c8-4c7d-4792-87ab-d7fc2487bee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:57.252629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8821c8-4c7d-4792-87ab-d7fc2487bee6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:57.253304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8821c8-4c7d-4792-87ab-d7fc2487bee6 None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=51de59caa3144a1cbf12b20b15e2134f) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:57.253685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8821c8-4c7d-4792-87ab-d7fc2487bee6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:57.256011 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8821c8-4c7d-4792-87ab-d7fc2487bee6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:57.259037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d0a2f-97ab-4129-b644-c3f1d80fc3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:57.260067 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d0a2f-97ab-4129-b644-c3f1d80fc3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:57.260498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d0a2f-97ab-4129-b644-c3f1d80fc3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:57.260954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d0a2f-97ab-4129-b644-c3f1d80fc3f9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:57.261796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d0a2f-97ab-4129-b644-c3f1d80fc3f9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:57.262471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 353/713] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:00:57 2022] GET /identity/v3/role_assignments?user.id=51de59caa3144a1cbf12b20b15e2134f&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:57.263612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d0a2f-97ab-4129-b644-c3f1d80fc3f9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:57.267930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8821c8-4c7d-4792-87ab-d7fc2487bee6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-892d0a2f-97ab-4129-b644-c3f1d80fc3f9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:57.280758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:57.282148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:57.283310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:57.283310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:57.283310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:57.283310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:57.283310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:57.283310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:57.283310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:57.283310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:57.283310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:57.283310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:57.283310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:57.283310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:00:57.283310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:00:57.283310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 15:00:57.283310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:57.286751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 361/714] 10.222.0.67 () {56 vars in 1129 bytes} [Fri Jun 24 15:00:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:57.293181 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d0a2f-97ab-4129-b644-c3f1d80fc3f9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:57.412791 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d0a2f-97ab-4129-b644-c3f1d80fc3f9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:57.487665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f09f89-4495-4814-980c-0f823906153f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:57.488372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f09f89-4495-4814-980c-0f823906153f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:57.488657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f09f89-4495-4814-980c-0f823906153f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:57.488939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f09f89-4495-4814-980c-0f823906153f None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:57.489857 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f09f89-4495-4814-980c-0f823906153f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:57.490167 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f09f89-4495-4814-980c-0f823906153f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:57.491559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f09f89-4495-4814-980c-0f823906153f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:57.494665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8821c8-4c7d-4792-87ab-d7fc2487bee6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:57.505943 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 362/715] 10.222.0.67 () {56 vars in 1124 bytes} [Fri Jun 24 15:00:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:57.510846 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f09f89-4495-4814-980c-0f823906153f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:57.552847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:57.561405 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10b076-12bd-437d-a347-546e29595f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:57.568341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10b076-12bd-437d-a347-546e29595f96 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:57.568341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10b076-12bd-437d-a347-546e29595f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:57.568341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10b076-12bd-437d-a347-546e29595f96 None admin] PATH_INFO: `/v3/services/network` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:57.568341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d10b076-12bd-437d-a347-546e29595f96 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:57.568341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d10b076-12bd-437d-a347-546e29595f96 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:57.573811 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-7d10b076-12bd-437d-a347-546e29595f96 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:57.592046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:57.593320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:57.594524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:57.594524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:57.594524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:57.594524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:57.594524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:57.594524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:57.594524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:57.594524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:57.594524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:57.594524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:57.594524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:00:57.594524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:00:57.594524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 15:00:57.594524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:57.594524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 354/716] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:00:57 2022] GET /identity/v3/services/network => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:57.600084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10b076-12bd-437d-a347-546e29595f96 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:57.694520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f09f89-4495-4814-980c-0f823906153f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:57.745758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685c50b-49cc-4b9c-ab4c-79b1812dae3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:57.746457 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685c50b-49cc-4b9c-ab4c-79b1812dae3e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:57.746747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685c50b-49cc-4b9c-ab4c-79b1812dae3e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:57.747029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685c50b-49cc-4b9c-ab4c-79b1812dae3e None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:57.747785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685c50b-49cc-4b9c-ab4c-79b1812dae3e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:57.748089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685c50b-49cc-4b9c-ab4c-79b1812dae3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:57.749478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685c50b-49cc-4b9c-ab4c-79b1812dae3e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:57.756641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 363/717] 10.222.0.67 () {56 vars in 1124 bytes} [Fri Jun 24 15:00:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:57.787375 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10b076-12bd-437d-a347-546e29595f96 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:57.838402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd95db-cc10-4fb2-877b-76bdbcfa2716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:57.839445 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd95db-cc10-4fb2-877b-76bdbcfa2716 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:57.839945 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd95db-cc10-4fb2-877b-76bdbcfa2716 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:57.840403 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd95db-cc10-4fb2-877b-76bdbcfa2716 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:57.841326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd95db-cc10-4fb2-877b-76bdbcfa2716 None admin] RBAC: Adding query filter params (name=network) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:57.841829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd95db-cc10-4fb2-877b-76bdbcfa2716 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:57.843471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd95db-cc10-4fb2-877b-76bdbcfa2716 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:57.851999 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 355/718] 10.222.0.67 () {56 vars in 1112 bytes} [Fri Jun 24 15:00:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:57.857689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685c50b-49cc-4b9c-ab4c-79b1812dae3e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:57.994714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685c50b-49cc-4b9c-ab4c-79b1812dae3e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:58.020691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:58.031538 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb90c25-d480-4512-82cf-0cdda70af09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:58.032213 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb90c25-d480-4512-82cf-0cdda70af09f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:58.032500 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb90c25-d480-4512-82cf-0cdda70af09f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:58.032781 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb90c25-d480-4512-82cf-0cdda70af09f None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:58.033503 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb90c25-d480-4512-82cf-0cdda70af09f None admin] RBAC: Adding query filter params (type=network) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:58.033846 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb90c25-d480-4512-82cf-0cdda70af09f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:58.035941 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb90c25-d480-4512-82cf-0cdda70af09f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:58.041238 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 364/719] 10.222.0.67 () {56 vars in 1112 bytes} [Fri Jun 24 15:00:57 2022] GET /identity/v3/services?type=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:00:59.223685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060117a6-9654-4cc3-9ff3-3f544dec6361 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:59.224227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060117a6-9654-4cc3-9ff3-3f544dec6361 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:59.224227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060117a6-9654-4cc3-9ff3-3f544dec6361 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:59.225910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 356/720] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:59.230688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed6c91-70ba-4066-89a2-831572833e24 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:59.231075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed6c91-70ba-4066-89a2-831572833e24 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:59.231265 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed6c91-70ba-4066-89a2-831572833e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:59.256173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7aed6c91-70ba-4066-89a2-831572833e24 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:59.257371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7aed6c91-70ba-4066-89a2-831572833e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:59.357923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 365/721] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:59.403370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060117a6-9654-4cc3-9ff3-3f544dec6361 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:59.485560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:59.543614 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060117a6-9654-4cc3-9ff3-3f544dec6361 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:59.576475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028e74d-5024-4e24-924f-d8f5a8f76151 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:59.577090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028e74d-5024-4e24-924f-d8f5a8f76151 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:59.577584 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028e74d-5024-4e24-924f-d8f5a8f76151 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:59.579377 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 366/722] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:00:59.585056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6f0b6-2d13-42ad-a479-bd611fe9f075 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:59.585624 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6f0b6-2d13-42ad-a479-bd611fe9f075 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:59.586098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6f0b6-2d13-42ad-a479-bd611fe9f075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:59.590571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9649cc5-909a-43b6-916a-dcb8395ae75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:59.591402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9649cc5-909a-43b6-916a-dcb8395ae75c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:59.591688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9649cc5-909a-43b6-916a-dcb8395ae75c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:59.591970 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9649cc5-909a-43b6-916a-dcb8395ae75c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:59.604454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ccb6f0b6-2d13-42ad-a479-bd611fe9f075 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:59.605906 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ccb6f0b6-2d13-42ad-a479-bd611fe9f075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:59.616153 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9649cc5-909a-43b6-916a-dcb8395ae75c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-e9649cc5-909a-43b6-916a-dcb8395ae75c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:59.623796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:59.624726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:59.625618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:59.625618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:59.625618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:59.625618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:59.625618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:59.625618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:59.625618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:59.625618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:59.625618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:59.625618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:59.625618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:00:59.625618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:00:59.630141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 357/723] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:00:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:59.631361 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9649cc5-909a-43b6-916a-dcb8395ae75c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:59.712445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 367/724] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 15:00:59.752448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28d971-92da-47b1-a583-f6f7c08ed993 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:59.753095 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28d971-92da-47b1-a583-f6f7c08ed993 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:59.753566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28d971-92da-47b1-a583-f6f7c08ed993 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:59.755145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 368/725] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:00:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:00:59.759333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28d971-92da-47b1-a583-f6f7c08ed993 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:59.781252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9649cc5-909a-43b6-916a-dcb8395ae75c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:59.798674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:59.803380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:59.818217 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982d920-3f4f-4026-ba5d-51ff0b1a9809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:59.819297 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982d920-3f4f-4026-ba5d-51ff0b1a9809 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:59.819856 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982d920-3f4f-4026-ba5d-51ff0b1a9809 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:59.820364 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982d920-3f4f-4026-ba5d-51ff0b1a9809 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:59.821341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982d920-3f4f-4026-ba5d-51ff0b1a9809 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:00:59.821881 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982d920-3f4f-4026-ba5d-51ff0b1a9809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:59.823516 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982d920-3f4f-4026-ba5d-51ff0b1a9809 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:59.833897 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 358/726] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:00:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:00:59.835444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08048b32-3ba1-459b-bc92-a7b299248bd3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:59.836026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08048b32-3ba1-459b-bc92-a7b299248bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:59.836498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08048b32-3ba1-459b-bc92-a7b299248bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:59.854147 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-08048b32-3ba1-459b-bc92-a7b299248bd3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:00:59.855638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-08048b32-3ba1-459b-bc92-a7b299248bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:59.874713 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28d971-92da-47b1-a583-f6f7c08ed993 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:59.912858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2862336-0651-42bc-9de3-9c33f57a1e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:59.913666 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2862336-0651-42bc-9de3-9c33f57a1e1e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:59.913951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2862336-0651-42bc-9de3-9c33f57a1e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:59.914228 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2862336-0651-42bc-9de3-9c33f57a1e1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:59.925344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2862336-0651-42bc-9de3-9c33f57a1e1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:00:59.930723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 359/727] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:00:59.931878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08048b32-3ba1-459b-bc92-a7b299248bd3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a2862336-0651-42bc-9de3-9c33f57a1e1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:00:59.933763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:00:59.934685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:00:59.935554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:00:59.935554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:00:59.935554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:00:59.935554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:00:59.935554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:00:59.935554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:00:59.935554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:00:59.935554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:00:59.935554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:00:59.935554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:00:59.935554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:00:59.935554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:00:59.935554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 369/728] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:00:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:00:59.944488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2862336-0651-42bc-9de3-9c33f57a1e1e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:00.056647 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08048b32-3ba1-459b-bc92-a7b299248bd3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:00.072057 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:00.072652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:00.079403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2862336-0651-42bc-9de3-9c33f57a1e1e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:00.099465 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e0655f-a988-45df-b355-2d000a9a3900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:00.100562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0655f-a988-45df-b355-2d000a9a3900 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:00.101043 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0655f-a988-45df-b355-2d000a9a3900 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:00.101981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0655f-a988-45df-b355-2d000a9a3900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:00.115830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e0655f-a988-45df-b355-2d000a9a3900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:00.118511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e0655f-a988-45df-b355-2d000a9a3900 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:00.122402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d917a50-29f5-4d56-bd96-f5418dd97344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:00.123205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d917a50-29f5-4d56-bd96-f5418dd97344 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:00.123556 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d917a50-29f5-4d56-bd96-f5418dd97344 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:00.123906 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d917a50-29f5-4d56-bd96-f5418dd97344 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:00.124789 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d917a50-29f5-4d56-bd96-f5418dd97344 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:00.125114 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d917a50-29f5-4d56-bd96-f5418dd97344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:00.126559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d917a50-29f5-4d56-bd96-f5418dd97344 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:00.137079 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 370/729] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:00:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:00.138013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d917a50-29f5-4d56-bd96-f5418dd97344 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:00.143493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-91e0655f-a988-45df-b355-2d000a9a3900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:00.143493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:00.143493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:00.143493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:00.143493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:00.143493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:00.143493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:00.143493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:00.143493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:00.143493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:00.143493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:00.143493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:00.143493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:00.143493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:00.143493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:00.144957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:00.146074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:00.146074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:00.146074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:00.146074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:00.148613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 360/730] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:00.149548 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e0655f-a988-45df-b355-2d000a9a3900 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:00.271908 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e0655f-a988-45df-b355-2d000a9a3900 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:00.275973 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d917a50-29f5-4d56-bd96-f5418dd97344 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:00.309544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58504402-98c8-48d8-8e1e-de2da0558829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:00.310460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58504402-98c8-48d8-8e1e-de2da0558829 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:00.310807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58504402-98c8-48d8-8e1e-de2da0558829 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:00.311135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58504402-98c8-48d8-8e1e-de2da0558829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:00.318357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90406ab-ff9e-41c1-82a1-13e729176258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:00.319779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90406ab-ff9e-41c1-82a1-13e729176258 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:00.320210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90406ab-ff9e-41c1-82a1-13e729176258 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:00.320649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90406ab-ff9e-41c1-82a1-13e729176258 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:00.323327 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58504402-98c8-48d8-8e1e-de2da0558829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:00.324821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58504402-98c8-48d8-8e1e-de2da0558829 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:00.334030 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90406ab-ff9e-41c1-82a1-13e729176258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a90406ab-ff9e-41c1-82a1-13e729176258 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:01:00.342498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:00.343647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:00.344687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:00.344687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:00.344687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:00.344687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:00.344687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:01:00.344687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:01:00.344687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:01:00.344687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:01:00.344687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:01:00.344687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:01:00.344687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:01:00.344687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:00.346112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-58504402-98c8-48d8-8e1e-de2da0558829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:00.346112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:00.346112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:00.346112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:00.346112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:00.346112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:00.346112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:00.346112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:00.346112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:00.346112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:00.346112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:00.346112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:00.346112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:00.346112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:00.346112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:00.347124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:00.348940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:00.348940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:00.348940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:00.348940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:00.350413 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 371/732] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:01:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:00.350576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 361/732] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:00.351412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58504402-98c8-48d8-8e1e-de2da0558829 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:00.356539 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90406ab-ff9e-41c1-82a1-13e729176258 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:00.377637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:00.379777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:00.469068 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90406ab-ff9e-41c1-82a1-13e729176258 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:00.484557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58504402-98c8-48d8-8e1e-de2da0558829 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:00.505128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64312d4e-abcd-44e8-a7d2-4b6c9995e5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:00.505801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64312d4e-abcd-44e8-a7d2-4b6c9995e5aa None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:00.506087 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64312d4e-abcd-44e8-a7d2-4b6c9995e5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:00.506362 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64312d4e-abcd-44e8-a7d2-4b6c9995e5aa None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:00.507110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64312d4e-abcd-44e8-a7d2-4b6c9995e5aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:00.507450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64312d4e-abcd-44e8-a7d2-4b6c9995e5aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:00.508956 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64312d4e-abcd-44e8-a7d2-4b6c9995e5aa None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:00.521037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 372/733] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:00.521469 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1ce95-5ad3-4703-90f7-e58632b0e2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:00.521830 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64312d4e-abcd-44e8-a7d2-4b6c9995e5aa None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:00.522200 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1ce95-5ad3-4703-90f7-e58632b0e2ee None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:00.522426 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1ce95-5ad3-4703-90f7-e58632b0e2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:00.522704 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1ce95-5ad3-4703-90f7-e58632b0e2ee None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:00.523466 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1ce95-5ad3-4703-90f7-e58632b0e2ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:00.523778 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1ce95-5ad3-4703-90f7-e58632b0e2ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:00.525398 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1ce95-5ad3-4703-90f7-e58632b0e2ee None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:00.537704 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 362/734] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:00.538969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea3624-3f07-4153-a5d3-c559e9856b2b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:00.539281 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea3624-3f07-4153-a5d3-c559e9856b2b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:00.539563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea3624-3f07-4153-a5d3-c559e9856b2b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:00.540788 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 363/735] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:00.541863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a07072-49c4-4ab1-906f-5e278bdded77 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:00.542130 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a07072-49c4-4ab1-906f-5e278bdded77 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:00.542413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a07072-49c4-4ab1-906f-5e278bdded77 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:00.543790 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 364/736] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:00.545305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a07072-49c4-4ab1-906f-5e278bdded77 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:00.609728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:00.624562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:00.636475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64312d4e-abcd-44e8-a7d2-4b6c9995e5aa None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:00.647576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a07072-49c4-4ab1-906f-5e278bdded77 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:00.665193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b0274-3039-4efa-81cd-6574a5b72646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:00.669145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b0274-3039-4efa-81cd-6574a5b72646 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:00.669145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b0274-3039-4efa-81cd-6574a5b72646 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:00.669145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b0274-3039-4efa-81cd-6574a5b72646 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:00.669145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b0274-3039-4efa-81cd-6574a5b72646 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:00.669145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b0274-3039-4efa-81cd-6574a5b72646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:00.669145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b0274-3039-4efa-81cd-6574a5b72646 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:00.677067 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 373/737] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:01:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:00.678093 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b0274-3039-4efa-81cd-6574a5b72646 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:00.681466 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436782cd-ed9a-4512-8bba-4291c2a1e802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:00.682264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436782cd-ed9a-4512-8bba-4291c2a1e802 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:00.682554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436782cd-ed9a-4512-8bba-4291c2a1e802 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:00.682835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436782cd-ed9a-4512-8bba-4291c2a1e802 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:00.690810 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436782cd-ed9a-4512-8bba-4291c2a1e802 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:00.692595 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436782cd-ed9a-4512-8bba-4291c2a1e802 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-436782cd-ed9a-4512-8bba-4291c2a1e802 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:01:00.708780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:00.709906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:00.710896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:00.710896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:00.710896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:01:00.710896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:01:00.710896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:01:00.710896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:01:00.710896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:01:00.710896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:01:00.710896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:01:00.710896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:00.711901 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 365/738] 10.222.0.67 () {56 vars in 1093 bytes} [Fri Jun 24 15:01:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:00.712949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5a734-bb08-4335-9a06-03ac09629d16 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:00.713236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5a734-bb08-4335-9a06-03ac09629d16 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:00.713510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5a734-bb08-4335-9a06-03ac09629d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:00.732621 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-46e5a734-bb08-4335-9a06-03ac09629d16 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:00.733854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-46e5a734-bb08-4335-9a06-03ac09629d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:00.806260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 366/739] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:01:00.807590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170f56d-5da3-49fa-87f0-efae7b43ef07 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:00.807879 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170f56d-5da3-49fa-87f0-efae7b43ef07 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:00.808160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170f56d-5da3-49fa-87f0-efae7b43ef07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:00.831013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5170f56d-5da3-49fa-87f0-efae7b43ef07 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:00.832398 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5170f56d-5da3-49fa-87f0-efae7b43ef07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:00.890617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:00.893626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b0274-3039-4efa-81cd-6574a5b72646 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:00.897848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 367/740] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 15:01:00.903687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170f56d-5da3-49fa-87f0-efae7b43ef07 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:00.930567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be37a31-0ad0-47ca-94b1-8955dd67306e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:00.931399 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be37a31-0ad0-47ca-94b1-8955dd67306e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:00.931684 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be37a31-0ad0-47ca-94b1-8955dd67306e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:00.931954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be37a31-0ad0-47ca-94b1-8955dd67306e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:00.938827 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be37a31-0ad0-47ca-94b1-8955dd67306e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:00.940288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be37a31-0ad0-47ca-94b1-8955dd67306e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:00.942736 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-5be37a31-0ad0-47ca-94b1-8955dd67306e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:01:00.955098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:00.956215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:00.957471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:00.957471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:00.957471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:01:00.957471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:01:00.957471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:01:00.957471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:01:00.957471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:01:00.957471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:01:00.957471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 15:01:00.957471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:00.957471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 374/741] 10.222.0.67 () {56 vars in 1117 bytes} [Fri Jun 24 15:01:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:00.957471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be37a31-0ad0-47ca-94b1-8955dd67306e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:01.003172 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170f56d-5da3-49fa-87f0-efae7b43ef07 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:01.036479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf24abc-1c31-4e1b-ab21-5b5b8c27ab4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:01.037261 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf24abc-1c31-4e1b-ab21-5b5b8c27ab4f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:01.037541 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf24abc-1c31-4e1b-ab21-5b5b8c27ab4f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:01.037813 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf24abc-1c31-4e1b-ab21-5b5b8c27ab4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:01.047594 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf24abc-1c31-4e1b-ab21-5b5b8c27ab4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:01.049028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf24abc-1c31-4e1b-ab21-5b5b8c27ab4f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:01.059223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be37a31-0ad0-47ca-94b1-8955dd67306e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:01.067462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-ddf24abc-1c31-4e1b-ab21-5b5b8c27ab4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:01.067462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:01.067462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:01.067462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:01.067462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:01.067462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:01.067462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:01.067462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:01.067462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:01.067462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:01.067462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:01.067462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:01.067462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:01.067462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:01.067462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:01.068349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:01.069272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:01.069272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:01.069272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:01.069272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:01.069272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 368/742] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:01.069550 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf24abc-1c31-4e1b-ab21-5b5b8c27ab4f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:01.093656 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09e581-67f0-47ac-a949-2eb0ba59b3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:01.094399 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09e581-67f0-47ac-a949-2eb0ba59b3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:01.094654 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09e581-67f0-47ac-a949-2eb0ba59b3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:01.094958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09e581-67f0-47ac-a949-2eb0ba59b3a7 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:01.095833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09e581-67f0-47ac-a949-2eb0ba59b3a7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:01.096144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09e581-67f0-47ac-a949-2eb0ba59b3a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:01.097701 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09e581-67f0-47ac-a949-2eb0ba59b3a7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:01.115429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 375/743] 10.222.0.67 () {56 vars in 1133 bytes} [Fri Jun 24 15:01:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:01.116218 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09e581-67f0-47ac-a949-2eb0ba59b3a7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:01.126157 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:01.171275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:01.176370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf24abc-1c31-4e1b-ab21-5b5b8c27ab4f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:01.209711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131ca42-cb5a-44f6-b797-7e685dd8eefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:01.210366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131ca42-cb5a-44f6-b797-7e685dd8eefb None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:01.210644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131ca42-cb5a-44f6-b797-7e685dd8eefb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:01.210915 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131ca42-cb5a-44f6-b797-7e685dd8eefb None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:01.211659 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131ca42-cb5a-44f6-b797-7e685dd8eefb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:01.211934 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09e581-67f0-47ac-a949-2eb0ba59b3a7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:01.213012 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131ca42-cb5a-44f6-b797-7e685dd8eefb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:01.244225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4653b4-fb21-4de2-812c-aedf37d789b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:01.244879 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4653b4-fb21-4de2-812c-aedf37d789b3 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:01.245159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4653b4-fb21-4de2-812c-aedf37d789b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:01.245432 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4653b4-fb21-4de2-812c-aedf37d789b3 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:01.246143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4653b4-fb21-4de2-812c-aedf37d789b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:01.247593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4653b4-fb21-4de2-812c-aedf37d789b3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:01.318716 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 369/744] 10.222.0.67 () {60 vars in 1141 bytes} [Fri Jun 24 15:01:01 2022] POST /identity/v3/services => generated 240 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:01:01.319644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131ca42-cb5a-44f6-b797-7e685dd8eefb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:01.343674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 376/745] 10.222.0.67 () {60 vars in 1141 bytes} [Fri Jun 24 15:01:01 2022] POST /identity/v3/services => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:01:01.343674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4653b4-fb21-4de2-812c-aedf37d789b3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:01.483422 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131ca42-cb5a-44f6-b797-7e685dd8eefb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:01.487814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:01.495599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4653b4-fb21-4de2-812c-aedf37d789b3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:01.518819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:01.532335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d05a1c-8817-4b62-917e-797cc1ee004e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:01.533021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d05a1c-8817-4b62-917e-797cc1ee004e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:01.533304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d05a1c-8817-4b62-917e-797cc1ee004e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:01.536319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d05a1c-8817-4b62-917e-797cc1ee004e None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:01.536319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d05a1c-8817-4b62-917e-797cc1ee004e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:01.536319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d05a1c-8817-4b62-917e-797cc1ee004e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:01.537158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d05a1c-8817-4b62-917e-797cc1ee004e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:01.543382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518983f-0be5-4341-bb71-e520311836f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:01.544054 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518983f-0be5-4341-bb71-e520311836f3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:01.544341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518983f-0be5-4341-bb71-e520311836f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:01.544629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518983f-0be5-4341-bb71-e520311836f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:01.545358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518983f-0be5-4341-bb71-e520311836f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:01.545668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518983f-0be5-4341-bb71-e520311836f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:01.547064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518983f-0be5-4341-bb71-e520311836f3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:01.562690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 377/746] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:01.563600 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518983f-0be5-4341-bb71-e520311836f3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:01.563801 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 370/747] 10.222.0.67 () {56 vars in 1149 bytes} [Fri Jun 24 15:01:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:01.569240 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d05a1c-8817-4b62-917e-797cc1ee004e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:01.670681 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518983f-0be5-4341-bb71-e520311836f3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:01.679027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d05a1c-8817-4b62-917e-797cc1ee004e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:01.706592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbcba4-a838-4466-939d-944de2215012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:01.708170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbcba4-a838-4466-939d-944de2215012 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:01.708769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbcba4-a838-4466-939d-944de2215012 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:01.709049 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbcba4-a838-4466-939d-944de2215012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:01.716219 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef347b-68a1-4716-8f27-15609d649be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:01.717060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef347b-68a1-4716-8f27-15609d649be8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:01.717419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef347b-68a1-4716-8f27-15609d649be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:01.717783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef347b-68a1-4716-8f27-15609d649be8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:01.721131 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbcba4-a838-4466-939d-944de2215012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:01.723627 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbcba4-a838-4466-939d-944de2215012 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:01.745859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-15dbcba4-a838-4466-939d-944de2215012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:01.745859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:01.745859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:01.745859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:01.745859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:01.745859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:01.745859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:01.745859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:01.745859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:01.745859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:01.745859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:01.745859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:01.745859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:01.745859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:01.745859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:01.746822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:01.747779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:01.747779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:01.747779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:01.747779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:01.751674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 378/748] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:01.752462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbcba4-a838-4466-939d-944de2215012 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:01.758000 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:01.782452 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:01.810243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef347b-68a1-4716-8f27-15609d649be8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:01.812401 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef347b-68a1-4716-8f27-15609d649be8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:01.862331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbcba4-a838-4466-939d-944de2215012 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:01.906183 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec787d9-aec6-4690-ba76-c4adc54602aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:01.907398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec787d9-aec6-4690-ba76-c4adc54602aa None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:01.907824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec787d9-aec6-4690-ba76-c4adc54602aa None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:01.908267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec787d9-aec6-4690-ba76-c4adc54602aa None admin] PATH_INFO: `/v3/users/swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:01.916959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec787d9-aec6-4690-ba76-c4adc54602aa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:01.918671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec787d9-aec6-4690-ba76-c4adc54602aa None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:01.935464 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 371/749] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:01:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 15:01:01.936556 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef347b-68a1-4716-8f27-15609d649be8 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4ec787d9-aec6-4690-ba76-c4adc54602aa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:01:01.937729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:01.938848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:01.939987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:01.939987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:01.939987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:01:01.939987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:01:01.939987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:01:01.939987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:01:01.939987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:01:01.939987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:01:01.939987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 15:01:01.939987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:01.943259 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 379/750] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:01:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:01.944284 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec787d9-aec6-4690-ba76-c4adc54602aa None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:02.022406 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:02.039010 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:02.059086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec787d9-aec6-4690-ba76-c4adc54602aa None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:02.066951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef347b-68a1-4716-8f27-15609d649be8 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:02.107679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c8f14-7220-4dcd-8216-d955c77c2fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:02.108688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c8f14-7220-4dcd-8216-d955c77c2fcc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:02.108977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c8f14-7220-4dcd-8216-d955c77c2fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:02.109256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c8f14-7220-4dcd-8216-d955c77c2fcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:02.110016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd4325-fca3-4748-a9a3-992040adf8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:02.110898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd4325-fca3-4748-a9a3-992040adf8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:02.111578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd4325-fca3-4748-a9a3-992040adf8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:02.112067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd4325-fca3-4748-a9a3-992040adf8c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:02.113020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd4325-fca3-4748-a9a3-992040adf8c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:02.113544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd4325-fca3-4748-a9a3-992040adf8c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:02.115137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd4325-fca3-4748-a9a3-992040adf8c6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:02.120867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c8f14-7220-4dcd-8216-d955c77c2fcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:02.122316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c8f14-7220-4dcd-8216-d955c77c2fcc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:02.128547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 372/751] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:02.130163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd4325-fca3-4748-a9a3-992040adf8c6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:02.150301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-7e2c8f14-7220-4dcd-8216-d955c77c2fcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:02.150301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:02.150301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:02.150301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:02.150301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:02.150301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:02.150301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:02.150301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:02.150301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:02.150301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:02.150301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:02.150301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:02.150301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:02.150301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:02.150301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:02.151606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:02.152490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:02.152490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:02.152490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:02.152490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:02.154539 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 380/752] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:02.155643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c8f14-7220-4dcd-8216-d955c77c2fcc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:02.258987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c8f14-7220-4dcd-8216-d955c77c2fcc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:02.269782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd4325-fca3-4748-a9a3-992040adf8c6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:02.300125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0411e3-688d-4726-b260-56b026375dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:02.301779 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:02.301987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0411e3-688d-4726-b260-56b026375dfe None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:02.302449 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0411e3-688d-4726-b260-56b026375dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:02.303275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0411e3-688d-4726-b260-56b026375dfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:02.312989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0411e3-688d-4726-b260-56b026375dfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:02.314225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0411e3-688d-4726-b260-56b026375dfe None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:02.315517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:02.317666 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0000891-135f-4795-9aaf-56eb15707836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:02.318507 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0000891-135f-4795-9aaf-56eb15707836 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:02.318881 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0000891-135f-4795-9aaf-56eb15707836 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:02.319484 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0000891-135f-4795-9aaf-56eb15707836 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:02.320237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0000891-135f-4795-9aaf-56eb15707836 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:02.320643 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0000891-135f-4795-9aaf-56eb15707836 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:02.322427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0000891-135f-4795-9aaf-56eb15707836 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:02.331741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-bd0411e3-688d-4726-b260-56b026375dfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:02.331741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:02.331741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:02.331741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:02.331741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:02.331741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:02.331741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:02.331741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:02.331741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:02.331741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:02.331741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:02.331741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:02.331741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:01:02.331741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:01:02.331741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:02.332634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:02.333530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:01:02.333530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:01:02.333530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:01:02.333530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:01:02.333530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:01:02.333530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:01:02.333530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:02.333530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:02.340931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 381/753] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:02.340931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0411e3-688d-4726-b260-56b026375dfe None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:02.350496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 373/754] 10.222.0.67 () {56 vars in 1135 bytes} [Fri Jun 24 15:01:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:02.351960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0000891-135f-4795-9aaf-56eb15707836 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:02.449916 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0411e3-688d-4726-b260-56b026375dfe None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:02.474750 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0000891-135f-4795-9aaf-56eb15707836 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:02.496045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06fb01d-7b3d-454f-9041-ef4fe98d412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:02.497259 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06fb01d-7b3d-454f-9041-ef4fe98d412e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:02.497549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06fb01d-7b3d-454f-9041-ef4fe98d412e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:02.498119 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06fb01d-7b3d-454f-9041-ef4fe98d412e None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:02.498859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06fb01d-7b3d-454f-9041-ef4fe98d412e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:02.499422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06fb01d-7b3d-454f-9041-ef4fe98d412e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:02.500809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06fb01d-7b3d-454f-9041-ef4fe98d412e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:02.518577 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 382/755] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:02.518577 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06fb01d-7b3d-454f-9041-ef4fe98d412e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:02.522880 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cb328-ef40-4c13-851a-2a4c3a9e36f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:02.523806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cb328-ef40-4c13-851a-2a4c3a9e36f6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:02.524242 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cb328-ef40-4c13-851a-2a4c3a9e36f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:02.524680 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cb328-ef40-4c13-851a-2a4c3a9e36f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:02.525627 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cb328-ef40-4c13-851a-2a4c3a9e36f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:02.526089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cb328-ef40-4c13-851a-2a4c3a9e36f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:02.528797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cb328-ef40-4c13-851a-2a4c3a9e36f6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:02.546547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 374/756] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:01:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:02.548120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cb328-ef40-4c13-851a-2a4c3a9e36f6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:02.606831 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:02.607616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:02.624171 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06fb01d-7b3d-454f-9041-ef4fe98d412e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:02.669569 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0fb4c-727d-421d-8b39-61b422004b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:02.670801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0fb4c-727d-421d-8b39-61b422004b78 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:02.671542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0fb4c-727d-421d-8b39-61b422004b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:02.672419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0fb4c-727d-421d-8b39-61b422004b78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:02.678783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cb328-ef40-4c13-851a-2a4c3a9e36f6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:02.713804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e7c9b-0f38-4cb4-97c3-55036475a2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:02.714409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e7c9b-0f38-4cb4-97c3-55036475a2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:02.714743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e7c9b-0f38-4cb4-97c3-55036475a2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:02.715019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e7c9b-0f38-4cb4-97c3-55036475a2d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:02.721915 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e7c9b-0f38-4cb4-97c3-55036475a2d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:02.723587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e7c9b-0f38-4cb4-97c3-55036475a2d3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:02.739296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-045e7c9b-0f38-4cb4-97c3-55036475a2d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:02.739296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:02.739296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:02.739296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:02.739296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:02.739296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:02.739296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:02.739296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:02.739296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:02.739296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:02.739296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:02.739296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:02.739296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:01:02.739296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:01:02.739296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:02.746031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:02.746961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:01:02.746961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:01:02.746961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:01:02.746961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:01:02.746961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:01:02.746961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:01:02.746961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:02.746961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:02.746961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 375/757] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:02.746961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e7c9b-0f38-4cb4-97c3-55036475a2d3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:02.768394 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0fb4c-727d-421d-8b39-61b422004b78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:02.770843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0fb4c-727d-421d-8b39-61b422004b78 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:02.872634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:02.877694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:02.881789 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e7c9b-0f38-4cb4-97c3-55036475a2d3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:02.929064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4864ca-f5cb-498e-b81d-ace13eb28106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:02.930653 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4864ca-f5cb-498e-b81d-ace13eb28106 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:02.931146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4864ca-f5cb-498e-b81d-ace13eb28106 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:02.931686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4864ca-f5cb-498e-b81d-ace13eb28106 None admin] PATH_INFO: `/v3/projects/06da054e8b7545dfa56780ab56585be8/users/51de59caa3144a1cbf12b20b15e2134f/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:02.935898 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 383/758] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:01:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:01:02.936872 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0fb4c-727d-421d-8b39-61b422004b78 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:02.937298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4864ca-f5cb-498e-b81d-ace13eb28106 None admin] RBAC: Authorizing `identity:create_grant(project_id=06da054e8b7545dfa56780ab56585be8, user_id=51de59caa3144a1cbf12b20b15e2134f, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:02.939034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4864ca-f5cb-498e-b81d-ace13eb28106 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:03.007773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 376/759] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:01:02 2022] PUT /identity/v3/projects/06da054e8b7545dfa56780ab56585be8/users/51de59caa3144a1cbf12b20b15e2134f/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:01:03.009374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4864ca-f5cb-498e-b81d-ace13eb28106 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:03.058102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0fb4c-727d-421d-8b39-61b422004b78 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:03.147789 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020e17e-8a06-49c3-8abc-b4f3c495b79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:03.148469 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020e17e-8a06-49c3-8abc-b4f3c495b79f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:03.148747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020e17e-8a06-49c3-8abc-b4f3c495b79f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:03.149019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020e17e-8a06-49c3-8abc-b4f3c495b79f None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:03.149819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020e17e-8a06-49c3-8abc-b4f3c495b79f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:03.150126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020e17e-8a06-49c3-8abc-b4f3c495b79f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:03.151621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020e17e-8a06-49c3-8abc-b4f3c495b79f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:03.152950 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4864ca-f5cb-498e-b81d-ace13eb28106 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:03.175143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 384/760] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:01:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:03.182718 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020e17e-8a06-49c3-8abc-b4f3c495b79f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:03.207631 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:03.209255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac357ee-ef95-4c0c-a174-96fc76edb3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:03.210299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac357ee-ef95-4c0c-a174-96fc76edb3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:03.210778 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac357ee-ef95-4c0c-a174-96fc76edb3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:03.211808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac357ee-ef95-4c0c-a174-96fc76edb3b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:03.225918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac357ee-ef95-4c0c-a174-96fc76edb3b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:03.227733 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac357ee-ef95-4c0c-a174-96fc76edb3b0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:03.250890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2ac357ee-ef95-4c0c-a174-96fc76edb3b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:03.250890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:03.250890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:03.250890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:03.250890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:03.250890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:03.250890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:03.250890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:03.250890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:03.250890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:03.250890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:03.250890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:03.250890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:03.250890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:03.250890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:03.251926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:03.252869 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:03.252869 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:03.252869 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:03.252869 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:03.256006 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 377/761] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:03.261131 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac357ee-ef95-4c0c-a174-96fc76edb3b0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:03.278818 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:03.322577 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020e17e-8a06-49c3-8abc-b4f3c495b79f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:03.364640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d701676-32ad-4a85-98e5-5c4b9f523cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:03.366353 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d701676-32ad-4a85-98e5-5c4b9f523cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:03.366980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d701676-32ad-4a85-98e5-5c4b9f523cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:03.368683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d701676-32ad-4a85-98e5-5c4b9f523cc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:03.369714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d701676-32ad-4a85-98e5-5c4b9f523cc1 None admin] RBAC: Adding query filter params (role.id=716cc96fb28146b7b9ae6271569a206f, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=72f00ac4c6a247e485ac52a8dee4f1cb) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:03.370995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d701676-32ad-4a85-98e5-5c4b9f523cc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:03.372642 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d701676-32ad-4a85-98e5-5c4b9f523cc1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:03.391625 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 385/762] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:01:03 2022] GET /identity/v3/role_assignments?user.id=72f00ac4c6a247e485ac52a8dee4f1cb&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=716cc96fb28146b7b9ae6271569a206f => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:03.450698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac357ee-ef95-4c0c-a174-96fc76edb3b0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:03.502968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db27b4-1777-4d9a-a3b8-229d19bfc481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:03.504254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db27b4-1777-4d9a-a3b8-229d19bfc481 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:03.504649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db27b4-1777-4d9a-a3b8-229d19bfc481 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:03.504911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db27b4-1777-4d9a-a3b8-229d19bfc481 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:03.505650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db27b4-1777-4d9a-a3b8-229d19bfc481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:03.505958 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db27b4-1777-4d9a-a3b8-229d19bfc481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:03.507413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db27b4-1777-4d9a-a3b8-229d19bfc481 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:03.525922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 378/763] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:03.531645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d701676-32ad-4a85-98e5-5c4b9f523cc1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:03.576780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd02956-e0ed-4cc0-b71e-40609191dedb None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:03.577067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd02956-e0ed-4cc0-b71e-40609191dedb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:03.577354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd02956-e0ed-4cc0-b71e-40609191dedb None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:03.578726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 379/764] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:03.586127 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca1fbb-ecdf-4295-9a61-40e61278039f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:03.586416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca1fbb-ecdf-4295-9a61-40e61278039f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:03.586693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca1fbb-ecdf-4295-9a61-40e61278039f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:03.618153 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4bca1fbb-ecdf-4295-9a61-40e61278039f None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:03.623220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4bca1fbb-ecdf-4295-9a61-40e61278039f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:03.679331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:03.685071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d701676-32ad-4a85-98e5-5c4b9f523cc1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:03.693546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 380/765] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:03.694895 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaa345-0196-4667-a0b0-3191396c7e1d None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:03.695228 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaa345-0196-4667-a0b0-3191396c7e1d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:03.695492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaa345-0196-4667-a0b0-3191396c7e1d None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:03.699576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 381/766] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:03.707482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e7c36-44b1-4563-902c-b6702954808f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:03.707713 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e7c36-44b1-4563-902c-b6702954808f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:03.707995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e7c36-44b1-4563-902c-b6702954808f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:03.717069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:03.736493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4f3e7c36-44b1-4563-902c-b6702954808f None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:03.739827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4f3e7c36-44b1-4563-902c-b6702954808f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:03.741506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf4f84-e879-4cdf-8c49-a1609c1534f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:03.742485 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf4f84-e879-4cdf-8c49-a1609c1534f8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:03.742865 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf4f84-e879-4cdf-8c49-a1609c1534f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:03.743291 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf4f84-e879-4cdf-8c49-a1609c1534f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:03.750809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf4f84-e879-4cdf-8c49-a1609c1534f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:03.752475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf4f84-e879-4cdf-8c49-a1609c1534f8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:03.767205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-3ddf4f84-e879-4cdf-8c49-a1609c1534f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:03.767205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:03.767205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:03.767205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:03.767205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:03.767205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:03.767205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:03.767205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:03.767205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:03.767205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:03.767205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:03.767205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:03.767205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:01:03.767205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:01:03.767205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:03.774762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:03.775809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:01:03.775809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:01:03.775809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:01:03.775809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:01:03.775809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:01:03.775809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:01:03.775809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:03.775809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:03.775809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 386/767] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:03.775809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf4f84-e879-4cdf-8c49-a1609c1534f8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:03.821377 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 382/768] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:03.822534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e7c36-44b1-4563-902c-b6702954808f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:03.899523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf4f84-e879-4cdf-8c49-a1609c1534f8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:03.931100 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e7c36-44b1-4563-902c-b6702954808f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:03.939622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274381a-d5d4-463c-9806-e431be662c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:03.941431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274381a-d5d4-463c-9806-e431be662c5d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:03.941929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274381a-d5d4-463c-9806-e431be662c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:03.942838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274381a-d5d4-463c-9806-e431be662c5d None admin] PATH_INFO: `/v3/services/network` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:03.943723 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274381a-d5d4-463c-9806-e431be662c5d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:03.945967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274381a-d5d4-463c-9806-e431be662c5d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f274381a-d5d4-463c-9806-e431be662c5d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:03.962842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:03.963926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:03.964793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:03.964793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:03.964793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:03.964793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:03.964793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:03.964793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:03.964793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:03.964793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:03.964793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:03.964793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:03.964793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:03.964793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:03.964793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 15:01:03.964793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:03.965541 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:03.968045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 387/769] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:03 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:03.969078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274381a-d5d4-463c-9806-e431be662c5d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:03.975907 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed025f-69cb-428f-8d08-28f7731ad5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:03.976844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed025f-69cb-428f-8d08-28f7731ad5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:03.977297 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed025f-69cb-428f-8d08-28f7731ad5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:03.977758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed025f-69cb-428f-8d08-28f7731ad5d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:03.978566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed025f-69cb-428f-8d08-28f7731ad5d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:03.982232 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed025f-69cb-428f-8d08-28f7731ad5d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:03.982232 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed025f-69cb-428f-8d08-28f7731ad5d4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:03.984182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:03.994159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 383/770] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:01:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:03.995278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed025f-69cb-428f-8d08-28f7731ad5d4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:04.114102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274381a-d5d4-463c-9806-e431be662c5d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:04.126960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed025f-69cb-428f-8d08-28f7731ad5d4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:04.149170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fdefa-956c-42e0-a0fd-b7053c5aa7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:04.150279 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fdefa-956c-42e0-a0fd-b7053c5aa7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:04.150790 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fdefa-956c-42e0-a0fd-b7053c5aa7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:04.151332 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fdefa-956c-42e0-a0fd-b7053c5aa7d7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:04.152197 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31fdefa-956c-42e0-a0fd-b7053c5aa7d7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:04.153760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31fdefa-956c-42e0-a0fd-b7053c5aa7d7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:04.164993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21376ce5-e195-41bb-aacb-216862f05a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:04.165662 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21376ce5-e195-41bb-aacb-216862f05a5e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:04.165952 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21376ce5-e195-41bb-aacb-216862f05a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:04.166233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21376ce5-e195-41bb-aacb-216862f05a5e None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:04.166944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21376ce5-e195-41bb-aacb-216862f05a5e None admin] RBAC: Adding query filter params (name=network) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:04.167279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21376ce5-e195-41bb-aacb-216862f05a5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-b31fdefa-956c-42e0-a0fd-b7053c5aa7d7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:04.168317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:04.169206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:04.170053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:04.170053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:04.170053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:04.170053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:04.170053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:04.170053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:04.170053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:04.170053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:04.170053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:04.170053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:04.170053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:04.170053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:04.170053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:04.170053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 15:01:04.170053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:04.170844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21376ce5-e195-41bb-aacb-216862f05a5e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:04.173054 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 388/771] 10.222.0.67 () {56 vars in 1129 bytes} [Fri Jun 24 15:01:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:04.174087 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fdefa-956c-42e0-a0fd-b7053c5aa7d7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:04.179189 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 384/772] 10.222.0.67 () {56 vars in 1112 bytes} [Fri Jun 24 15:01:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:04.179719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21376ce5-e195-41bb-aacb-216862f05a5e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:04.242469 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:04.248416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:04.281774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fdefa-956c-42e0-a0fd-b7053c5aa7d7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:04.292203 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21376ce5-e195-41bb-aacb-216862f05a5e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:04.318910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c92d9e-5032-4bbd-8e99-67e9ba9874f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:04.319943 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c92d9e-5032-4bbd-8e99-67e9ba9874f7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:04.320428 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c92d9e-5032-4bbd-8e99-67e9ba9874f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:04.320903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c92d9e-5032-4bbd-8e99-67e9ba9874f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:04.321956 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c92d9e-5032-4bbd-8e99-67e9ba9874f7 None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=e4732ab3c8c446958092246d16b4b4f0) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:04.322472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c92d9e-5032-4bbd-8e99-67e9ba9874f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:04.324130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c92d9e-5032-4bbd-8e99-67e9ba9874f7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:04.331595 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea7aeb-309a-47c3-8d50-9535019c27d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:04.332356 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea7aeb-309a-47c3-8d50-9535019c27d3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:04.332642 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea7aeb-309a-47c3-8d50-9535019c27d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:04.332927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea7aeb-309a-47c3-8d50-9535019c27d3 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:04.333635 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea7aeb-309a-47c3-8d50-9535019c27d3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:04.334051 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea7aeb-309a-47c3-8d50-9535019c27d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:04.334529 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 389/773] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:01:04 2022] GET /identity/v3/role_assignments?user.id=e4732ab3c8c446958092246d16b4b4f0&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:04.335470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea7aeb-309a-47c3-8d50-9535019c27d3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:04.335881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c92d9e-5032-4bbd-8e99-67e9ba9874f7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:04.346198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 385/774] 10.222.0.67 () {56 vars in 1124 bytes} [Fri Jun 24 15:01:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:04.347905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea7aeb-309a-47c3-8d50-9535019c27d3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:04.470591 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea7aeb-309a-47c3-8d50-9535019c27d3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:04.494861 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:04.512367 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee9e9f-50b8-45c9-b03a-e06070db4a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:04.513046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee9e9f-50b8-45c9-b03a-e06070db4a6a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:04.513333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee9e9f-50b8-45c9-b03a-e06070db4a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:04.513615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee9e9f-50b8-45c9-b03a-e06070db4a6a None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:04.514443 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ee9e9f-50b8-45c9-b03a-e06070db4a6a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:04.514757 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ee9e9f-50b8-45c9-b03a-e06070db4a6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:04.516190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ee9e9f-50b8-45c9-b03a-e06070db4a6a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:04.523084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 386/775] 10.222.0.67 () {56 vars in 1124 bytes} [Fri Jun 24 15:01:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:04.529145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee9e9f-50b8-45c9-b03a-e06070db4a6a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:04.548462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c92d9e-5032-4bbd-8e99-67e9ba9874f7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:04.609043 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3ae9a-ed60-4ddb-9b9e-cee3b238c6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:04.609711 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3ae9a-ed60-4ddb-9b9e-cee3b238c6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:04.609997 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3ae9a-ed60-4ddb-9b9e-cee3b238c6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:04.610275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3ae9a-ed60-4ddb-9b9e-cee3b238c6b3 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:04.611044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3ae9a-ed60-4ddb-9b9e-cee3b238c6b3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:04.611381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3ae9a-ed60-4ddb-9b9e-cee3b238c6b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:04.614648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3ae9a-ed60-4ddb-9b9e-cee3b238c6b3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:04.616771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:04.627835 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 390/776] 10.222.0.67 () {56 vars in 1112 bytes} [Fri Jun 24 15:01:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:04.638602 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3ae9a-ed60-4ddb-9b9e-cee3b238c6b3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:04.675845 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee9e9f-50b8-45c9-b03a-e06070db4a6a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:04.748974 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf69631-3dda-47ba-8600-6278aac1131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:04.750501 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf69631-3dda-47ba-8600-6278aac1131b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:04.750788 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf69631-3dda-47ba-8600-6278aac1131b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:04.751069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf69631-3dda-47ba-8600-6278aac1131b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:04.751894 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf69631-3dda-47ba-8600-6278aac1131b None admin] RBAC: Adding query filter params (interface=public, service_id=0f1508f3de0a4b3189da60c15c629e96, region_id=RegionOne) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:04.752208 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf69631-3dda-47ba-8600-6278aac1131b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:04.753654 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf69631-3dda-47ba-8600-6278aac1131b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:04.774203 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 387/777] 10.222.0.67 () {56 vars in 1251 bytes} [Fri Jun 24 15:01:04 2022] GET /identity/v3/endpoints?service_id=0f1508f3de0a4b3189da60c15c629e96&interface=public®ion_id=RegionOne => generated 194 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:04.779903 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf69631-3dda-47ba-8600-6278aac1131b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:04.835752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3ae9a-ed60-4ddb-9b9e-cee3b238c6b3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:04.861064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:04.897067 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266742e-6713-48e6-8f65-7861cbbea0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:04.898156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266742e-6713-48e6-8f65-7861cbbea0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:04.898681 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266742e-6713-48e6-8f65-7861cbbea0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:04.898958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266742e-6713-48e6-8f65-7861cbbea0f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:04.899848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266742e-6713-48e6-8f65-7861cbbea0f0 None admin] RBAC: Adding query filter params (interface=public, service_id=e390c97f72a247cabec5f6654fc19245, region_id=RegionOne) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:04.900144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266742e-6713-48e6-8f65-7861cbbea0f0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:04.901506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266742e-6713-48e6-8f65-7861cbbea0f0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:04.909893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 391/778] 10.222.0.67 () {56 vars in 1251 bytes} [Fri Jun 24 15:01:04 2022] GET /identity/v3/endpoints?service_id=e390c97f72a247cabec5f6654fc19245&interface=public®ion_id=RegionOne => generated 194 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:04.917913 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266742e-6713-48e6-8f65-7861cbbea0f0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:04.942604 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf69631-3dda-47ba-8600-6278aac1131b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:04.983311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cc7d6-9554-4600-9b2e-f25a977232b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:04.984458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cc7d6-9554-4600-9b2e-f25a977232b5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:04.986724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cc7d6-9554-4600-9b2e-f25a977232b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:04.987220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cc7d6-9554-4600-9b2e-f25a977232b5 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:04.988329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cc7d6-9554-4600-9b2e-f25a977232b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:04.990648 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cc7d6-9554-4600-9b2e-f25a977232b5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:05.000693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 388/779] 10.222.0.67 () {56 vars in 1087 bytes} [Fri Jun 24 15:01:04 2022] GET /identity/v3/services => generated 772 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:05.041929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:05.104865 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266742e-6713-48e6-8f65-7861cbbea0f0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:05.163308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69218b29-1575-4228-af4c-1d654c80a412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:05.163994 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69218b29-1575-4228-af4c-1d654c80a412 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:05.164283 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69218b29-1575-4228-af4c-1d654c80a412 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:05.164567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69218b29-1575-4228-af4c-1d654c80a412 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:05.165266 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69218b29-1575-4228-af4c-1d654c80a412 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:05.166650 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69218b29-1575-4228-af4c-1d654c80a412 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:05.174057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 392/780] 10.222.0.67 () {56 vars in 1087 bytes} [Fri Jun 24 15:01:04 2022] GET /identity/v3/services => generated 772 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:05.555537 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62691078-4e41-46e3-82e3-3141d9c00fab None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:05.556588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62691078-4e41-46e3-82e3-3141d9c00fab None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:05.557082 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62691078-4e41-46e3-82e3-3141d9c00fab None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:05.558915 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 389/781] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:05.570009 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc6c6a-a1fa-4f6d-98f0-b9a8b6f65e21 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:05.570929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc6c6a-a1fa-4f6d-98f0-b9a8b6f65e21 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:05.571460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc6c6a-a1fa-4f6d-98f0-b9a8b6f65e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:05.594688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6acc6c6a-a1fa-4f6d-98f0-b9a8b6f65e21 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:05.596253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6acc6c6a-a1fa-4f6d-98f0-b9a8b6f65e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:05.630733 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438aec6e-d938-46cc-baed-a3a18c0a9e6b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:05.631049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438aec6e-d938-46cc-baed-a3a18c0a9e6b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:05.631340 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438aec6e-d938-46cc-baed-a3a18c0a9e6b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:05.632919 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 390/782] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:05.638100 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e14893-c105-41be-a72f-dad397f454c4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:05.638384 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e14893-c105-41be-a72f-dad397f454c4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:05.638661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e14893-c105-41be-a72f-dad397f454c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:05.657037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c3e14893-c105-41be-a72f-dad397f454c4 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:05.658573 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c3e14893-c105-41be-a72f-dad397f454c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:05.683209 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 393/783] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:05.721075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc6c6a-a1fa-4f6d-98f0-b9a8b6f65e21 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:05.757069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 391/784] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:05.801982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:05.824932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e14893-c105-41be-a72f-dad397f454c4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:05.873950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc6c6a-a1fa-4f6d-98f0-b9a8b6f65e21 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:05.885585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:05.943731 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e14893-c105-41be-a72f-dad397f454c4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:05.961215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc4696-e8bf-4ef2-a87a-c75aa4b946f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:05.962028 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc4696-e8bf-4ef2-a87a-c75aa4b946f5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:05.962311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc4696-e8bf-4ef2-a87a-c75aa4b946f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:05.962586 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc4696-e8bf-4ef2-a87a-c75aa4b946f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:05.984705 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26cfad-be58-415d-9c62-0dbaed0082ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:05.985509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26cfad-be58-415d-9c62-0dbaed0082ae None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:05.985790 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26cfad-be58-415d-9c62-0dbaed0082ae None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:05.986064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26cfad-be58-415d-9c62-0dbaed0082ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:05.991068 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc4696-e8bf-4ef2-a87a-c75aa4b946f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-41cc4696-e8bf-4ef2-a87a-c75aa4b946f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:01:06.002927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:06.005177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:06.006074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:06.006074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:06.006074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:06.006074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:06.006074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:01:06.006074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:01:06.006074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:01:06.006074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:01:06.006074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:01:06.006074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:01:06.006074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:01:06.006074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:06.006074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 394/785] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:01:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:06.016576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26cfad-be58-415d-9c62-0dbaed0082ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-ef26cfad-be58-415d-9c62-0dbaed0082ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:01:06.025199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:06.027437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:06.029544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:06.029544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:06.029544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:06.029544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:06.029544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:01:06.029544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:01:06.029544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:01:06.029544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:01:06.029544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:01:06.029544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:01:06.029544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:01:06.029544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:06.032646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc4696-e8bf-4ef2-a87a-c75aa4b946f5 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:06.032844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 392/786] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:01:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:06.042310 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26cfad-be58-415d-9c62-0dbaed0082ae None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:06.212191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc4696-e8bf-4ef2-a87a-c75aa4b946f5 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:06.229886 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:06.251746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26cfad-be58-415d-9c62-0dbaed0082ae None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:06.259140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6caa60-5901-488c-a026-216a697fdecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:06.259611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:06.260642 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6caa60-5901-488c-a026-216a697fdecf None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:06.263265 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6caa60-5901-488c-a026-216a697fdecf None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:06.263801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6caa60-5901-488c-a026-216a697fdecf None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:06.264769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6caa60-5901-488c-a026-216a697fdecf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:06.265301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6caa60-5901-488c-a026-216a697fdecf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:06.266954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6caa60-5901-488c-a026-216a697fdecf None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:06.287580 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 395/787] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:01:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:06.292438 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6caa60-5901-488c-a026-216a697fdecf None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:06.323978 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd609e52-7458-4e6e-8b6f-840c45ec235c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:06.325030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd609e52-7458-4e6e-8b6f-840c45ec235c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:06.325532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd609e52-7458-4e6e-8b6f-840c45ec235c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:06.326033 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd609e52-7458-4e6e-8b6f-840c45ec235c None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:06.327001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd609e52-7458-4e6e-8b6f-840c45ec235c None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:06.328995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd609e52-7458-4e6e-8b6f-840c45ec235c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:06.330599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd609e52-7458-4e6e-8b6f-840c45ec235c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:06.348197 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 393/788] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:01:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:06.353720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd609e52-7458-4e6e-8b6f-840c45ec235c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:06.491814 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6caa60-5901-488c-a026-216a697fdecf None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:06.526804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8fd06-d9a4-4d06-a446-2396605dc3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:06.528013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8fd06-d9a4-4d06-a446-2396605dc3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:06.528494 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8fd06-d9a4-4d06-a446-2396605dc3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:06.528965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8fd06-d9a4-4d06-a446-2396605dc3b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:06.540499 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8fd06-d9a4-4d06-a446-2396605dc3b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:06.542203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8fd06-d9a4-4d06-a446-2396605dc3b0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:06.546346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd609e52-7458-4e6e-8b6f-840c45ec235c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:06.560517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f1a8fd06-d9a4-4d06-a446-2396605dc3b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:06.560517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:06.560517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:06.560517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:06.560517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:06.560517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:06.560517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:06.560517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:06.560517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:06.560517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:06.560517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:06.560517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:06.560517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:06.560517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:06.560517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:06.561448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:06.562364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:06.562364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:06.562364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:06.562364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:06.569451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 396/789] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:06.570972 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8fd06-d9a4-4d06-a446-2396605dc3b0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:06.619674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1a60a-e504-4066-9c73-2e90906c8983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:06.621155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1a60a-e504-4066-9c73-2e90906c8983 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:06.621638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1a60a-e504-4066-9c73-2e90906c8983 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:06.623559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1a60a-e504-4066-9c73-2e90906c8983 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:06.649011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:06.649940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1a60a-e504-4066-9c73-2e90906c8983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:06.651676 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1a60a-e504-4066-9c73-2e90906c8983 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:06.689251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-17b1a60a-e504-4066-9c73-2e90906c8983 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:06.689251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:06.689251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:06.689251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:06.689251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:06.689251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:06.689251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:06.689251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:06.689251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:06.689251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:06.689251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:06.689251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:06.689251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:06.689251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:06.689251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:06.690303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:06.692030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:06.692030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:06.692030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:06.692030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:06.700797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 394/790] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:06.705982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1a60a-e504-4066-9c73-2e90906c8983 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:06.742374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8fd06-d9a4-4d06-a446-2396605dc3b0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:06.754111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:06.777493 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42254f51-bc53-4ff0-a144-89ac1b4dfee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:06.778289 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42254f51-bc53-4ff0-a144-89ac1b4dfee7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:06.778644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42254f51-bc53-4ff0-a144-89ac1b4dfee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:06.778986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42254f51-bc53-4ff0-a144-89ac1b4dfee7 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:06.779789 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42254f51-bc53-4ff0-a144-89ac1b4dfee7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:06.780161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42254f51-bc53-4ff0-a144-89ac1b4dfee7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:06.781602 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42254f51-bc53-4ff0-a144-89ac1b4dfee7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:06.798457 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 397/791] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:06.799931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4832dd3-70aa-4501-adbf-9dc3d02291fe None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:06.800221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4832dd3-70aa-4501-adbf-9dc3d02291fe None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:06.801115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4832dd3-70aa-4501-adbf-9dc3d02291fe None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:06.803098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 398/792] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:06.804036 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4832dd3-70aa-4501-adbf-9dc3d02291fe None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:06.861920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1a60a-e504-4066-9c73-2e90906c8983 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:06.908482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12414181-c303-45f0-8c26-cef5300cd54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:06.909153 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12414181-c303-45f0-8c26-cef5300cd54b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:06.910507 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12414181-c303-45f0-8c26-cef5300cd54b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:06.910793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12414181-c303-45f0-8c26-cef5300cd54b None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:06.911530 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12414181-c303-45f0-8c26-cef5300cd54b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:06.911833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12414181-c303-45f0-8c26-cef5300cd54b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:06.913331 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12414181-c303-45f0-8c26-cef5300cd54b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:06.916386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:06.925142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 395/793] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:06.927565 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bef19c-4eea-4ba2-971d-1cff7fe7d7bd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:06.927863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bef19c-4eea-4ba2-971d-1cff7fe7d7bd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:06.928138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bef19c-4eea-4ba2-971d-1cff7fe7d7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:06.929697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4832dd3-70aa-4501-adbf-9dc3d02291fe None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:06.945023 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-17bef19c-4eea-4ba2-971d-1cff7fe7d7bd None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:06.946303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-17bef19c-4eea-4ba2-971d-1cff7fe7d7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:06.967669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353ae32-0c46-4e1f-bb54-018e081513e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:06.968783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353ae32-0c46-4e1f-bb54-018e081513e6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:06.969295 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353ae32-0c46-4e1f-bb54-018e081513e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:06.969774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353ae32-0c46-4e1f-bb54-018e081513e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:06.978669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5353ae32-0c46-4e1f-bb54-018e081513e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:06.980443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5353ae32-0c46-4e1f-bb54-018e081513e6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-5353ae32-0c46-4e1f-bb54-018e081513e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:01:06.996433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:06.997409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:06.998370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:06.998370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:06.998370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:01:06.998370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:01:06.998370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:01:06.998370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:01:06.998370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:01:06.998370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:01:06.998370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:01:06.998370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:07.000788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 399/794] 10.222.0.67 () {56 vars in 1093 bytes} [Fri Jun 24 15:01:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:07.001784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353ae32-0c46-4e1f-bb54-018e081513e6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:07.025239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 396/795] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:07.026343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bef19c-4eea-4ba2-971d-1cff7fe7d7bd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:07.035333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:07.116340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353ae32-0c46-4e1f-bb54-018e081513e6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:07.147156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bef19c-4eea-4ba2-971d-1cff7fe7d7bd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:07.153946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8771262-a4e2-4176-a1b0-06a1a0702e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:07.154908 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8771262-a4e2-4176-a1b0-06a1a0702e88 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:07.155349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8771262-a4e2-4176-a1b0-06a1a0702e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:07.155728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8771262-a4e2-4176-a1b0-06a1a0702e88 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:07.165067 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8771262-a4e2-4176-a1b0-06a1a0702e88 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:07.166545 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8771262-a4e2-4176-a1b0-06a1a0702e88 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e8771262-a4e2-4176-a1b0-06a1a0702e88 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:01:07.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:07.184878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:07.186026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:07.186026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:07.186026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:01:07.186026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:01:07.186026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:01:07.186026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:01:07.186026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:01:07.186026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:01:07.186026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 15:01:07.186026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:07.186026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 400/796] 10.222.0.67 () {56 vars in 1117 bytes} [Fri Jun 24 15:01:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:07.187151 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8771262-a4e2-4176-a1b0-06a1a0702e88 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:07.189344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241fada-a93c-47d1-afab-6e88959b06c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:07.190694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241fada-a93c-47d1-afab-6e88959b06c0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:07.191886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241fada-a93c-47d1-afab-6e88959b06c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:07.192353 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241fada-a93c-47d1-afab-6e88959b06c0 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:07.193468 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241fada-a93c-47d1-afab-6e88959b06c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:07.193954 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241fada-a93c-47d1-afab-6e88959b06c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:07.195679 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241fada-a93c-47d1-afab-6e88959b06c0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:07.201182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:07.216532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 397/797] 10.222.0.67 () {56 vars in 1133 bytes} [Fri Jun 24 15:01:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:07.221253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241fada-a93c-47d1-afab-6e88959b06c0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:07.291244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:07.297705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8771262-a4e2-4176-a1b0-06a1a0702e88 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:07.333032 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241fada-a93c-47d1-afab-6e88959b06c0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:07.333494 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdadb3f-d5c2-4ee7-931b-403e0d1bb8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:07.334378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdadb3f-d5c2-4ee7-931b-403e0d1bb8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:07.334737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdadb3f-d5c2-4ee7-931b-403e0d1bb8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:07.335087 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdadb3f-d5c2-4ee7-931b-403e0d1bb8e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:07.345746 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdadb3f-d5c2-4ee7-931b-403e0d1bb8e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-7fdadb3f-d5c2-4ee7-931b-403e0d1bb8e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:01:07.354566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:07.355560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:07.356598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:07.356598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:07.356598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:07.356598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:07.356598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:01:07.356598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:01:07.356598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:01:07.356598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:01:07.356598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:01:07.356598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:01:07.356598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:01:07.356598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:07.356598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 401/798] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:01:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:07.358162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdadb3f-d5c2-4ee7-931b-403e0d1bb8e8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:07.374708 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d914394-6894-4d3b-bebb-328e1dbdf6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:07.375691 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d914394-6894-4d3b-bebb-328e1dbdf6ba None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:07.376136 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d914394-6894-4d3b-bebb-328e1dbdf6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:07.376555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d914394-6894-4d3b-bebb-328e1dbdf6ba None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:07.377669 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d914394-6894-4d3b-bebb-328e1dbdf6ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:07.378304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d914394-6894-4d3b-bebb-328e1dbdf6ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:07.380755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d914394-6894-4d3b-bebb-328e1dbdf6ba None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:07.401732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 398/799] 10.222.0.67 () {56 vars in 1149 bytes} [Fri Jun 24 15:01:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:07.402962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d914394-6894-4d3b-bebb-328e1dbdf6ba None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:07.432547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:07.466131 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdadb3f-d5c2-4ee7-931b-403e0d1bb8e8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:07.507637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085dd8be-becf-42d0-b36f-876552ab1169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:07.508479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085dd8be-becf-42d0-b36f-876552ab1169 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:07.508864 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085dd8be-becf-42d0-b36f-876552ab1169 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:07.509239 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085dd8be-becf-42d0-b36f-876552ab1169 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:07.586071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085dd8be-becf-42d0-b36f-876552ab1169 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:07.587721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085dd8be-becf-42d0-b36f-876552ab1169 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:07.645132 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d914394-6894-4d3b-bebb-328e1dbdf6ba None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:07.656856 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:07.667761 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:07.679625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25276ba-07f4-4c88-9449-564361af3cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:07.680296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25276ba-07f4-4c88-9449-564361af3cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:07.680578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25276ba-07f4-4c88-9449-564361af3cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:07.680891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25276ba-07f4-4c88-9449-564361af3cc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:07.681640 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25276ba-07f4-4c88-9449-564361af3cc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:07.681950 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25276ba-07f4-4c88-9449-564361af3cc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:07.683399 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25276ba-07f4-4c88-9449-564361af3cc5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:07.686623 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 402/800] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:01:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:01:07.687536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085dd8be-becf-42d0-b36f-876552ab1169 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:07.700027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 399/801] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:01:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:07.700027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787b339-f112-499c-b45c-f5f1a8e4859a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:07.700027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787b339-f112-499c-b45c-f5f1a8e4859a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:07.700027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787b339-f112-499c-b45c-f5f1a8e4859a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:07.700027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 400/802] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:07.700507 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d70228-e70d-4cab-9f79-166191444ed6 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:07.700631 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d70228-e70d-4cab-9f79-166191444ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:07.700972 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d70228-e70d-4cab-9f79-166191444ed6 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:07.702305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 401/803] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:07.702857 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d70228-e70d-4cab-9f79-166191444ed6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:07.788932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085dd8be-becf-42d0-b36f-876552ab1169 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:07.809486 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d70228-e70d-4cab-9f79-166191444ed6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:07.822728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7d266-cdf8-47b2-b54c-0634b02e8263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:07.823557 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7d266-cdf8-47b2-b54c-0634b02e8263 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:07.823841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7d266-cdf8-47b2-b54c-0634b02e8263 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:07.824119 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7d266-cdf8-47b2-b54c-0634b02e8263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:07.834762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7d266-cdf8-47b2-b54c-0634b02e8263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:07.836239 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7d266-cdf8-47b2-b54c-0634b02e8263 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:07.842276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0817fe-bc77-49fe-abab-fdd0891be725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:07.843049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0817fe-bc77-49fe-abab-fdd0891be725 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:07.843370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0817fe-bc77-49fe-abab-fdd0891be725 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:07.843648 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0817fe-bc77-49fe-abab-fdd0891be725 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:07.854465 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0817fe-bc77-49fe-abab-fdd0891be725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:07.856139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0817fe-bc77-49fe-abab-fdd0891be725 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:07.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e0d7d266-cdf8-47b2-b54c-0634b02e8263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:07.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:07.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:07.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:07.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:07.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:07.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:07.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:07.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:07.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:07.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:07.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:07.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:07.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:07.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:07.858070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:07.859157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:07.859157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:07.859157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:07.859157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:07.859157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 403/804] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:07.859836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7d266-cdf8-47b2-b54c-0634b02e8263 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:07.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-8a0817fe-bc77-49fe-abab-fdd0891be725 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:07.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:07.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:07.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:07.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:07.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:07.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:07.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:07.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:07.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:07.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:07.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:07.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:07.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:07.877430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:07.878444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:07.879863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:07.879863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:07.879863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:07.879863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:07.879863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 402/805] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:07.880623 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a8b35-0c0b-4411-be1a-e80f3caa4353 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:07.880912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a8b35-0c0b-4411-be1a-e80f3caa4353 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:07.881192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a8b35-0c0b-4411-be1a-e80f3caa4353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:07.891291 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:07.898201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-222a8b35-0c0b-4411-be1a-e80f3caa4353 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:07.904009 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-222a8b35-0c0b-4411-be1a-e80f3caa4353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:07.928875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:07.966145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 403/806] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:07.966843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7d266-cdf8-47b2-b54c-0634b02e8263 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:07.967607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182639b4-a0b9-4d81-9b6f-f1bd86b4d74e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:07.967997 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182639b4-a0b9-4d81-9b6f-f1bd86b4d74e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:07.968379 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182639b4-a0b9-4d81-9b6f-f1bd86b4d74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:07.985084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-182639b4-a0b9-4d81-9b6f-f1bd86b4d74e None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:07.986262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-182639b4-a0b9-4d81-9b6f-f1bd86b4d74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:08.002300 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0b54f-eb6f-4c8a-8b51-4b347edb8bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:08.003098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0b54f-eb6f-4c8a-8b51-4b347edb8bba None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:08.003477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0b54f-eb6f-4c8a-8b51-4b347edb8bba None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:08.003758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0b54f-eb6f-4c8a-8b51-4b347edb8bba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:08.013630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a0b54f-eb6f-4c8a-8b51-4b347edb8bba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:08.015063 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a0b54f-eb6f-4c8a-8b51-4b347edb8bba None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:08.033102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-92a0b54f-eb6f-4c8a-8b51-4b347edb8bba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:08.033102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:08.033102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:08.033102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:08.033102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:08.033102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:08.033102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:08.033102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:08.033102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:08.033102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:08.033102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:08.033102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:08.033102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:08.033102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:08.033102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:08.034222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:08.036249 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:08.036249 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:08.036249 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:08.036249 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:08.036249 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 404/807] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:08.036249 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0b54f-eb6f-4c8a-8b51-4b347edb8bba None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:08.055171 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 404/808] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:08.056028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182639b4-a0b9-4d81-9b6f-f1bd86b4d74e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:08.115891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:08.141235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0b54f-eb6f-4c8a-8b51-4b347edb8bba None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:08.154111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:08.160219 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182639b4-a0b9-4d81-9b6f-f1bd86b4d74e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:08.173907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579d5a3-57e6-4090-904a-01b36fc52495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:08.174561 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579d5a3-57e6-4090-904a-01b36fc52495 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:08.174840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579d5a3-57e6-4090-904a-01b36fc52495 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:08.175113 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579d5a3-57e6-4090-904a-01b36fc52495 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:08.176083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579d5a3-57e6-4090-904a-01b36fc52495 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:08.176306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579d5a3-57e6-4090-904a-01b36fc52495 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:08.178151 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579d5a3-57e6-4090-904a-01b36fc52495 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:08.189372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 405/809] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:08.189976 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579d5a3-57e6-4090-904a-01b36fc52495 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:08.199249 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ae4f3-5d50-4b17-9bcc-9db760eb1f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:08.199916 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ae4f3-5d50-4b17-9bcc-9db760eb1f1a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:08.200196 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ae4f3-5d50-4b17-9bcc-9db760eb1f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:08.200471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ae4f3-5d50-4b17-9bcc-9db760eb1f1a None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:08.201191 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ae4f3-5d50-4b17-9bcc-9db760eb1f1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:08.201588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ae4f3-5d50-4b17-9bcc-9db760eb1f1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:08.202930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ae4f3-5d50-4b17-9bcc-9db760eb1f1a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:08.220484 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 405/810] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:08.220484 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ae4f3-5d50-4b17-9bcc-9db760eb1f1a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:08.288131 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579d5a3-57e6-4090-904a-01b36fc52495 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:08.321455 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8975bd6-f9b6-4bf0-b8d7-b6780c5433a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:08.322253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8975bd6-f9b6-4bf0-b8d7-b6780c5433a6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:08.322489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ae4f3-5d50-4b17-9bcc-9db760eb1f1a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:08.322597 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8975bd6-f9b6-4bf0-b8d7-b6780c5433a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:08.322801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8975bd6-f9b6-4bf0-b8d7-b6780c5433a6 None admin] PATH_INFO: `/v3/services/network` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:08.323479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8975bd6-f9b6-4bf0-b8d7-b6780c5433a6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:08.324844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8975bd6-f9b6-4bf0-b8d7-b6780c5433a6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:08.326551 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d8975bd6-f9b6-4bf0-b8d7-b6780c5433a6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:08.337072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:08.343645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:08.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:08.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:08.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:08.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:08.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:08.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:08.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:08.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:08.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:08.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:08.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:08.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:08.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 15:01:08.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:08.344686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 406/811] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:08 2022] GET /identity/v3/services/network => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:08.345511 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8975bd6-f9b6-4bf0-b8d7-b6780c5433a6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:08.356028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91576932-bf5e-43ca-92f5-d74391eadf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:08.356694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91576932-bf5e-43ca-92f5-d74391eadf51 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:08.356977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91576932-bf5e-43ca-92f5-d74391eadf51 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:08.357255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91576932-bf5e-43ca-92f5-d74391eadf51 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:08.357973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91576932-bf5e-43ca-92f5-d74391eadf51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:08.358277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91576932-bf5e-43ca-92f5-d74391eadf51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:08.359670 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91576932-bf5e-43ca-92f5-d74391eadf51 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:08.372464 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 406/812] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:08.373251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91576932-bf5e-43ca-92f5-d74391eadf51 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:08.389854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:08.449376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8975bd6-f9b6-4bf0-b8d7-b6780c5433a6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:08.471161 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91576932-bf5e-43ca-92f5-d74391eadf51 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:08.482991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2476b-6613-4a53-85e0-ef623b753eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:08.483827 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2476b-6613-4a53-85e0-ef623b753eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:08.484119 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2476b-6613-4a53-85e0-ef623b753eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:08.484395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2476b-6613-4a53-85e0-ef623b753eb1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:08.485046 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2476b-6613-4a53-85e0-ef623b753eb1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:08.486688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2476b-6613-4a53-85e0-ef623b753eb1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-94e2476b-6613-4a53-85e0-ef623b753eb1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:08.499516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:08.500661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:08.501537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 407/813] 10.222.0.67 () {56 vars in 1129 bytes} [Fri Jun 24 15:01:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:08.502403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2476b-6613-4a53-85e0-ef623b753eb1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:08.509800 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e22b9-8c61-420b-bc48-7a5defb26a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:08.510599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e22b9-8c61-420b-bc48-7a5defb26a9a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:08.510877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e22b9-8c61-420b-bc48-7a5defb26a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:08.511148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e22b9-8c61-420b-bc48-7a5defb26a9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:08.517995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e22b9-8c61-420b-bc48-7a5defb26a9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:08.519963 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e22b9-8c61-420b-bc48-7a5defb26a9a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:08.533460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-657e22b9-8c61-420b-bc48-7a5defb26a9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:08.533460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:08.533460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:08.533460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:08.533460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:08.533460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:08.533460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:08.533460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:08.533460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:08.533460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:08.533460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:08.533460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:08.533460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:01:08.533460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:01:08.533460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:08.534609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:08.536305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:01:08.536305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:01:08.536305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:01:08.536305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:01:08.536305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:01:08.536305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:01:08.536305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:08.536305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:08.536305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 407/814] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:08.536305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e22b9-8c61-420b-bc48-7a5defb26a9a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:08.566901 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:08.607072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2476b-6613-4a53-85e0-ef623b753eb1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:08.610917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:08.633160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e22b9-8c61-420b-bc48-7a5defb26a9a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:08.641011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1496e2d-da15-4a60-90bc-c2f66bfb851e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:08.641809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1496e2d-da15-4a60-90bc-c2f66bfb851e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:08.642098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1496e2d-da15-4a60-90bc-c2f66bfb851e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:08.642378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1496e2d-da15-4a60-90bc-c2f66bfb851e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:08.650535 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1496e2d-da15-4a60-90bc-c2f66bfb851e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:08.652003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1496e2d-da15-4a60-90bc-c2f66bfb851e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:08.666916 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9a7cc-b041-4e8e-b497-69156641ac0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:08.667611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9a7cc-b041-4e8e-b497-69156641ac0f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:08.667876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d1496e2d-da15-4a60-90bc-c2f66bfb851e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:08.667876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:08.667876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:08.667876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:08.667876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:08.667876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:08.667876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:08.667876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:08.667876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:08.667876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:08.667876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:08.667876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:08.667876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:01:08.667876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:01:08.667876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:08.668873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:08.669826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:01:08.669826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:01:08.669826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:01:08.669826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:01:08.669826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:01:08.669826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:01:08.669826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:08.669826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:08.670261 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9a7cc-b041-4e8e-b497-69156641ac0f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:08.670261 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9a7cc-b041-4e8e-b497-69156641ac0f None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:08.670261 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9a7cc-b041-4e8e-b497-69156641ac0f None admin] RBAC: Adding query filter params (name=network) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:08.670441 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 408/815] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:08.670517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9a7cc-b041-4e8e-b497-69156641ac0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:08.670587 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1496e2d-da15-4a60-90bc-c2f66bfb851e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:08.670655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9a7cc-b041-4e8e-b497-69156641ac0f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:08.674812 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 408/816] 10.222.0.67 () {56 vars in 1112 bytes} [Fri Jun 24 15:01:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:08.676241 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9a7cc-b041-4e8e-b497-69156641ac0f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:08.775472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1496e2d-da15-4a60-90bc-c2f66bfb851e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:08.778473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9a7cc-b041-4e8e-b497-69156641ac0f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:08.797527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:08.809718 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1d160-8b4f-40ff-ac3c-7b9728c760dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:08.810505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1d160-8b4f-40ff-ac3c-7b9728c760dc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:08.810956 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1d160-8b4f-40ff-ac3c-7b9728c760dc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:08.811123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1d160-8b4f-40ff-ac3c-7b9728c760dc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:08.812310 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac6b02-4b8a-4e5c-9827-e2028161ae66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:08.812963 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac6b02-4b8a-4e5c-9827-e2028161ae66 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:08.813280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac6b02-4b8a-4e5c-9827-e2028161ae66 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:08.813566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac6b02-4b8a-4e5c-9827-e2028161ae66 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:08.814254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac6b02-4b8a-4e5c-9827-e2028161ae66 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:08.814554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac6b02-4b8a-4e5c-9827-e2028161ae66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:08.816503 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac6b02-4b8a-4e5c-9827-e2028161ae66 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:08.818642 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1d160-8b4f-40ff-ac3c-7b9728c760dc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:08.820130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1d160-8b4f-40ff-ac3c-7b9728c760dc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:08.821108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 409/817] 10.222.0.67 () {56 vars in 1124 bytes} [Fri Jun 24 15:01:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:08.821976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac6b02-4b8a-4e5c-9827-e2028161ae66 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:08.826631 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-6ba1d160-8b4f-40ff-ac3c-7b9728c760dc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:01:08.840158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:08.841245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:08.842783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:08.842783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:08.842783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:01:08.842783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:01:08.842783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:01:08.842783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:01:08.842783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:01:08.842783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:01:08.842783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 15:01:08.842783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:08.842783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 409/818] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:01:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:08.842783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1d160-8b4f-40ff-ac3c-7b9728c760dc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:08.922129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac6b02-4b8a-4e5c-9827-e2028161ae66 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:08.948578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1d160-8b4f-40ff-ac3c-7b9728c760dc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:08.954945 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b33480-47bb-4910-9dab-3ba32aad2e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:08.955623 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b33480-47bb-4910-9dab-3ba32aad2e86 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:08.955904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b33480-47bb-4910-9dab-3ba32aad2e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:08.956173 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b33480-47bb-4910-9dab-3ba32aad2e86 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:08.956943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b33480-47bb-4910-9dab-3ba32aad2e86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:08.957246 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b33480-47bb-4910-9dab-3ba32aad2e86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:08.958656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b33480-47bb-4910-9dab-3ba32aad2e86 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:08.970043 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 410/819] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:01:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:08.970814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b33480-47bb-4910-9dab-3ba32aad2e86 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:08.986942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18ce43-cdd9-41bc-ac48-807f479e1d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:08.988433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18ce43-cdd9-41bc-ac48-807f479e1d46 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:08.988732 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18ce43-cdd9-41bc-ac48-807f479e1d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:08.989017 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18ce43-cdd9-41bc-ac48-807f479e1d46 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:08.989813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18ce43-cdd9-41bc-ac48-807f479e1d46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:08.990127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18ce43-cdd9-41bc-ac48-807f479e1d46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:08.991592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18ce43-cdd9-41bc-ac48-807f479e1d46 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:09.004041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 410/820] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:01:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:09.004603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18ce43-cdd9-41bc-ac48-807f479e1d46 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:09.034313 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:09.048363 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:09.072232 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b33480-47bb-4910-9dab-3ba32aad2e86 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:09.106009 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18ce43-cdd9-41bc-ac48-807f479e1d46 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:09.107448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcb574-d03a-4722-af8e-0cab285d1d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:09.108112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcb574-d03a-4722-af8e-0cab285d1d5a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:09.108394 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcb574-d03a-4722-af8e-0cab285d1d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:09.108667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcb574-d03a-4722-af8e-0cab285d1d5a None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:09.109363 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcb574-d03a-4722-af8e-0cab285d1d5a None admin] RBAC: Adding query filter params (type=network) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:09.109669 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcb574-d03a-4722-af8e-0cab285d1d5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:09.111017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcb574-d03a-4722-af8e-0cab285d1d5a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:09.116396 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 411/821] 10.222.0.67 () {56 vars in 1112 bytes} [Fri Jun 24 15:01:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:09.117159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcb574-d03a-4722-af8e-0cab285d1d5a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:09.144057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee3b04-90c1-4a7b-8b88-9ce8f336daff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:09.144715 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee3b04-90c1-4a7b-8b88-9ce8f336daff None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:09.144997 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee3b04-90c1-4a7b-8b88-9ce8f336daff None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:09.145276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee3b04-90c1-4a7b-8b88-9ce8f336daff None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:09.145975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee3b04-90c1-4a7b-8b88-9ce8f336daff None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:09.146283 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee3b04-90c1-4a7b-8b88-9ce8f336daff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:09.147690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee3b04-90c1-4a7b-8b88-9ce8f336daff None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:09.152350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 411/822] 10.222.0.67 () {56 vars in 1124 bytes} [Fri Jun 24 15:01:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:09.153090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee3b04-90c1-4a7b-8b88-9ce8f336daff None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:09.216036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcb574-d03a-4722-af8e-0cab285d1d5a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:09.243115 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:09.250821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ed032-3f3f-4fc6-9b87-37316e22543e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:09.251358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee3b04-90c1-4a7b-8b88-9ce8f336daff None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:09.251521 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ed032-3f3f-4fc6-9b87-37316e22543e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:09.251797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ed032-3f3f-4fc6-9b87-37316e22543e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:09.252072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ed032-3f3f-4fc6-9b87-37316e22543e None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:09.252901 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ed032-3f3f-4fc6-9b87-37316e22543e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:09.253215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ed032-3f3f-4fc6-9b87-37316e22543e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:09.254666 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ed032-3f3f-4fc6-9b87-37316e22543e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:09.258987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:09.272495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 412/823] 10.222.0.67 () {56 vars in 1135 bytes} [Fri Jun 24 15:01:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:09.274245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ed032-3f3f-4fc6-9b87-37316e22543e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:09.292613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddc313-4e6c-4325-89c4-07da83c639e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:09.293448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddc313-4e6c-4325-89c4-07da83c639e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:09.293735 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddc313-4e6c-4325-89c4-07da83c639e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:09.294015 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddc313-4e6c-4325-89c4-07da83c639e4 None admin] PATH_INFO: `/v3/projects/06da054e8b7545dfa56780ab56585be8/users/72f00ac4c6a247e485ac52a8dee4f1cb/roles/716cc96fb28146b7b9ae6271569a206f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:09.308959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbddc313-4e6c-4325-89c4-07da83c639e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=06da054e8b7545dfa56780ab56585be8, user_id=72f00ac4c6a247e485ac52a8dee4f1cb, role_id=716cc96fb28146b7b9ae6271569a206f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:09.310404 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbddc313-4e6c-4325-89c4-07da83c639e4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:09.373162 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ed032-3f3f-4fc6-9b87-37316e22543e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:09.385475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 412/824] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:01:09 2022] PUT /identity/v3/projects/06da054e8b7545dfa56780ab56585be8/users/72f00ac4c6a247e485ac52a8dee4f1cb/roles/716cc96fb28146b7b9ae6271569a206f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:01:09.386251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddc313-4e6c-4325-89c4-07da83c639e4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:09.410993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baddddba-2cc0-43d5-a364-e6ec3df6ee5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:09.411723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddddba-2cc0-43d5-a364-e6ec3df6ee5f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:09.412026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddddba-2cc0-43d5-a364-e6ec3df6ee5f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:09.412342 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baddddba-2cc0-43d5-a364-e6ec3df6ee5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:09.413197 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baddddba-2cc0-43d5-a364-e6ec3df6ee5f None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=51de59caa3144a1cbf12b20b15e2134f) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:09.413519 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baddddba-2cc0-43d5-a364-e6ec3df6ee5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:09.414996 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baddddba-2cc0-43d5-a364-e6ec3df6ee5f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:09.422449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 413/825] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:01:09 2022] GET /identity/v3/role_assignments?user.id=51de59caa3144a1cbf12b20b15e2134f&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:09.423821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baddddba-2cc0-43d5-a364-e6ec3df6ee5f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:09.496410 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:09.565035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddc313-4e6c-4325-89c4-07da83c639e4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:09.585206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baddddba-2cc0-43d5-a364-e6ec3df6ee5f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:09.617232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbcb7a0-1262-43aa-896c-900e14778ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:09.618586 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcb7a0-1262-43aa-896c-900e14778ea9 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:09.619051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcb7a0-1262-43aa-896c-900e14778ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:09.619377 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbcb7a0-1262-43aa-896c-900e14778ea9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:09.620484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbcb7a0-1262-43aa-896c-900e14778ea9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:09.622302 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbcb7a0-1262-43aa-896c-900e14778ea9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:09.630809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5230ec9-63a0-40c0-8cff-903f41168d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:09.630809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5230ec9-63a0-40c0-8cff-903f41168d31 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:09.630809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5230ec9-63a0-40c0-8cff-903f41168d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:09.630809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5230ec9-63a0-40c0-8cff-903f41168d31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:09.631215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5230ec9-63a0-40c0-8cff-903f41168d31 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:09.631678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5230ec9-63a0-40c0-8cff-903f41168d31 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:09.741872 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 413/826] 10.222.0.67 () {60 vars in 1144 bytes} [Fri Jun 24 15:01:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:01:09.742742 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbcb7a0-1262-43aa-896c-900e14778ea9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:09.760970 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:09.772894 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 414/827] 10.222.0.67 () {60 vars in 1144 bytes} [Fri Jun 24 15:01:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:01:09.910745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbcb7a0-1262-43aa-896c-900e14778ea9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:09.984786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21763c-e153-4c25-8e3a-61f6a24f7e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:09.985865 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21763c-e153-4c25-8e3a-61f6a24f7e71 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:09.986298 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21763c-e153-4c25-8e3a-61f6a24f7e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:09.986720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21763c-e153-4c25-8e3a-61f6a24f7e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:09.999584 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21763c-e153-4c25-8e3a-61f6a24f7e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:10.001335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21763c-e153-4c25-8e3a-61f6a24f7e71 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:10.024381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-bc21763c-e153-4c25-8e3a-61f6a24f7e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:10.024381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:10.024381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:10.024381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:10.024381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:10.024381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:10.024381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:10.024381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:10.024381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:10.024381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:10.024381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:10.024381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:10.024381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:10.024381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:10.024381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:10.025534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:10.026678 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:10.026678 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:10.026678 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:10.026678 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:10.028322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 414/828] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:10.034171 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5230ec9-63a0-40c0-8cff-903f41168d31 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:10.148660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5230ec9-63a0-40c0-8cff-903f41168d31 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:10.156613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:10.186757 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b92c2-6de8-47a2-a89e-0802b086c697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:10.187411 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b92c2-6de8-47a2-a89e-0802b086c697 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:10.187411 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b92c2-6de8-47a2-a89e-0802b086c697 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:10.187560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b92c2-6de8-47a2-a89e-0802b086c697 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:10.188166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b92c2-6de8-47a2-a89e-0802b086c697 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:10.188471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b92c2-6de8-47a2-a89e-0802b086c697 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:10.189833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b92c2-6de8-47a2-a89e-0802b086c697 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:10.202169 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 415/829] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:10.208553 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21763c-e153-4c25-8e3a-61f6a24f7e71 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:10.267308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:10.313884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21763c-e153-4c25-8e3a-61f6a24f7e71 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:10.348981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f523a1-a833-4965-9c64-425e492768ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:10.349785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f523a1-a833-4965-9c64-425e492768ea None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:10.350073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f523a1-a833-4965-9c64-425e492768ea None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:10.350354 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f523a1-a833-4965-9c64-425e492768ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:10.357129 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f523a1-a833-4965-9c64-425e492768ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:10.358574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f523a1-a833-4965-9c64-425e492768ea None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:10.373027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-84f523a1-a833-4965-9c64-425e492768ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:10.373027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:10.373027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:10.373027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:10.373027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:10.373027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:10.373027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:10.373027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:10.373027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:10.373027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:10.373027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:10.373027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:10.373027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:01:10.373027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:01:10.373027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:10.373988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:10.375292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:01:10.375292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:01:10.375292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:01:10.375292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:01:10.375292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:01:10.375292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:01:10.375292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:10.375292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:10.375292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 415/830] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:10.382058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b92c2-6de8-47a2-a89e-0802b086c697 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:10.518118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b92c2-6de8-47a2-a89e-0802b086c697 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:10.553867 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529698b0-44db-4c4f-8a72-cf39fc1082b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:10.554529 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529698b0-44db-4c4f-8a72-cf39fc1082b5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:10.554817 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529698b0-44db-4c4f-8a72-cf39fc1082b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:10.555093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529698b0-44db-4c4f-8a72-cf39fc1082b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:10.555908 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529698b0-44db-4c4f-8a72-cf39fc1082b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:10.556220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529698b0-44db-4c4f-8a72-cf39fc1082b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:10.557640 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529698b0-44db-4c4f-8a72-cf39fc1082b5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:10.570866 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 416/831] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:01:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:10.577312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f523a1-a833-4965-9c64-425e492768ea None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:10.684829 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f523a1-a833-4965-9c64-425e492768ea None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:10.702444 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:10.721318 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494234d6-59e7-4f6b-8e1c-d96334651432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:10.728603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494234d6-59e7-4f6b-8e1c-d96334651432 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:10.728603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494234d6-59e7-4f6b-8e1c-d96334651432 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:10.728603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494234d6-59e7-4f6b-8e1c-d96334651432 None admin] PATH_INFO: `/v3/projects/06da054e8b7545dfa56780ab56585be8/users/e4732ab3c8c446958092246d16b4b4f0/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:10.728603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494234d6-59e7-4f6b-8e1c-d96334651432 None admin] RBAC: Authorizing `identity:create_grant(project_id=06da054e8b7545dfa56780ab56585be8, user_id=e4732ab3c8c446958092246d16b4b4f0, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:10.728603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494234d6-59e7-4f6b-8e1c-d96334651432 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:10.844080 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 416/832] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:01:10 2022] PUT /identity/v3/projects/06da054e8b7545dfa56780ab56585be8/users/e4732ab3c8c446958092246d16b4b4f0/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:01:11.636806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d4ef8-a2c3-439e-bdf8-23301d89560e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:11.637335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d4ef8-a2c3-439e-bdf8-23301d89560e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:11.637446 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d4ef8-a2c3-439e-bdf8-23301d89560e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:11.638980 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 417/833] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:11.644744 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b36af-2ed5-4f98-9a33-5b9c0422c230 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:11.645474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b36af-2ed5-4f98-9a33-5b9c0422c230 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:11.645968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b36af-2ed5-4f98-9a33-5b9c0422c230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:11.664368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4d8b36af-2ed5-4f98-9a33-5b9c0422c230 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:11.665818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4d8b36af-2ed5-4f98-9a33-5b9c0422c230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:11.738361 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 417/834] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:11 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:11.763737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d4ef8-a2c3-439e-bdf8-23301d89560e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:11.792182 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:11.871245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d4ef8-a2c3-439e-bdf8-23301d89560e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:11.906271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb3749-288d-41f1-a4f3-ba71cfb58195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:11.906995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb3749-288d-41f1-a4f3-ba71cfb58195 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:11.907309 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb3749-288d-41f1-a4f3-ba71cfb58195 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:11.907588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb3749-288d-41f1-a4f3-ba71cfb58195 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:11.918831 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb3749-288d-41f1-a4f3-ba71cfb58195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-e9cb3749-288d-41f1-a4f3-ba71cfb58195 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:01:11.928186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:11.929151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:11.930022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:11.930022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:11.930022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:11.930022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:11.930022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:01:11.930022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:01:11.930022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:01:11.930022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:01:11.930022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:01:11.930022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:01:11.930022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:01:11.930022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:11.930022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 418/835] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:01:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:11.936960 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b36af-2ed5-4f98-9a33-5b9c0422c230 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:12.044471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b36af-2ed5-4f98-9a33-5b9c0422c230 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:12.053499 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:12.082125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a098431-2d99-43d6-bed0-a9e26fc742a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:12.083168 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a098431-2d99-43d6-bed0-a9e26fc742a0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:12.083683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a098431-2d99-43d6-bed0-a9e26fc742a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:12.084180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a098431-2d99-43d6-bed0-a9e26fc742a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:12.085133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a098431-2d99-43d6-bed0-a9e26fc742a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:12.085659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a098431-2d99-43d6-bed0-a9e26fc742a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:12.087287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a098431-2d99-43d6-bed0-a9e26fc742a0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:12.096442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 418/836] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:01:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:12.103236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb3749-288d-41f1-a4f3-ba71cfb58195 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:12.193876 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:12.210487 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb3749-288d-41f1-a4f3-ba71cfb58195 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:12.247508 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f40e23-8c06-4804-9a76-64bc0b3b4750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:12.248485 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f40e23-8c06-4804-9a76-64bc0b3b4750 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:12.248776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f40e23-8c06-4804-9a76-64bc0b3b4750 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:12.249058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f40e23-8c06-4804-9a76-64bc0b3b4750 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:12.259791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f40e23-8c06-4804-9a76-64bc0b3b4750 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:12.261497 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f40e23-8c06-4804-9a76-64bc0b3b4750 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:12.279975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-45f40e23-8c06-4804-9a76-64bc0b3b4750 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:12.279975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:12.279975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:12.279975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:12.279975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:12.279975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:12.279975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:12.279975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:12.279975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:12.279975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:12.279975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:12.279975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:12.279975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:12.279975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:12.279975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:12.281629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:12.283407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:12.283407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:12.283407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:12.283407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:12.283407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 419/837] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:12.288596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a098431-2d99-43d6-bed0-a9e26fc742a0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:12.399094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a098431-2d99-43d6-bed0-a9e26fc742a0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:12.433955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a72ea-141e-4d6f-bdc3-3ad8271df719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:12.434626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a72ea-141e-4d6f-bdc3-3ad8271df719 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:12.434915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a72ea-141e-4d6f-bdc3-3ad8271df719 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:12.435205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a72ea-141e-4d6f-bdc3-3ad8271df719 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:12.435948 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a72ea-141e-4d6f-bdc3-3ad8271df719 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:12.436258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a72ea-141e-4d6f-bdc3-3ad8271df719 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:12.437631 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a72ea-141e-4d6f-bdc3-3ad8271df719 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:12.451359 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 419/838] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:12.456766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f40e23-8c06-4804-9a76-64bc0b3b4750 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:12.502547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4a616-1a6a-452d-9ed1-90e17db129e5 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:12.502976 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4a616-1a6a-452d-9ed1-90e17db129e5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:12.503201 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4a616-1a6a-452d-9ed1-90e17db129e5 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:12.505547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 420/839] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:12.512425 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8137c-dbae-425b-8e3f-920a8b206d91 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:12.512706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8137c-dbae-425b-8e3f-920a8b206d91 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:12.512984 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8137c-dbae-425b-8e3f-920a8b206d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:12.530085 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-25f8137c-dbae-425b-8e3f-920a8b206d91 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:12.531664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-25f8137c-dbae-425b-8e3f-920a8b206d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:12.557992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:12.578128 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f40e23-8c06-4804-9a76-64bc0b3b4750 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:12.615525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 421/840] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:12.618820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04361f4c-a079-4719-9715-9909954ba27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:12.619716 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04361f4c-a079-4719-9715-9909954ba27a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:12.619999 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04361f4c-a079-4719-9715-9909954ba27a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:12.620278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04361f4c-a079-4719-9715-9909954ba27a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:12.627778 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04361f4c-a079-4719-9715-9909954ba27a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:12.629212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04361f4c-a079-4719-9715-9909954ba27a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:12.630923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:12.635292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8137c-dbae-425b-8e3f-920a8b206d91 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-04361f4c-a079-4719-9715-9909954ba27a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:01:12.647638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:12.648961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:12.649993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:12.649993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:12.649993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:01:12.649993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:01:12.649993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:01:12.649993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:01:12.649993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:01:12.649993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:01:12.649993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 15:01:12.649993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:12.649993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 420/841] 10.222.0.67 () {56 vars in 1117 bytes} [Fri Jun 24 15:01:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:12.655032 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04361f4c-a079-4719-9715-9909954ba27a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:12.748705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8137c-dbae-425b-8e3f-920a8b206d91 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:12.783938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04361f4c-a079-4719-9715-9909954ba27a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:12.787668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d16a2-4f44-4197-8a0f-e7e54477801b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:12.788720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d16a2-4f44-4197-8a0f-e7e54477801b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:12.789221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d16a2-4f44-4197-8a0f-e7e54477801b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:12.789712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d16a2-4f44-4197-8a0f-e7e54477801b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:12.790673 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d16a2-4f44-4197-8a0f-e7e54477801b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:12.792344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d16a2-4f44-4197-8a0f-e7e54477801b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a94d16a2-4f44-4197-8a0f-e7e54477801b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:12.809675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:12.810677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:12.811568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:12.811568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:12.811568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:12.811568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:12.811568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:12.811568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:12.811568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:12.811568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:12.811568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:12.811568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:12.811568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:12.811568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:12.811568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:12.811568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 15:01:12.811568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:12.819521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 422/842] 10.222.0.67 () {56 vars in 1132 bytes} [Fri Jun 24 15:01:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:12.820911 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d16a2-4f44-4197-8a0f-e7e54477801b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:12.823250 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82beaa3d-eda3-4f2d-94d9-a64efc469fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:12.823932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82beaa3d-eda3-4f2d-94d9-a64efc469fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:12.824213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82beaa3d-eda3-4f2d-94d9-a64efc469fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:12.824483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82beaa3d-eda3-4f2d-94d9-a64efc469fc2 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:12.826511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82beaa3d-eda3-4f2d-94d9-a64efc469fc2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:12.826991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82beaa3d-eda3-4f2d-94d9-a64efc469fc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:12.828608 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82beaa3d-eda3-4f2d-94d9-a64efc469fc2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:12.852898 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:12.853147 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 421/843] 10.222.0.67 () {56 vars in 1149 bytes} [Fri Jun 24 15:01:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:12.854164 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82beaa3d-eda3-4f2d-94d9-a64efc469fc2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:12.913228 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:12.931068 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d16a2-4f44-4197-8a0f-e7e54477801b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:12.959169 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82beaa3d-eda3-4f2d-94d9-a64efc469fc2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:12.967511 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8a263-8b3f-46fb-8141-4dc239c1e061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:12.968184 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8a263-8b3f-46fb-8141-4dc239c1e061 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:12.968465 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8a263-8b3f-46fb-8141-4dc239c1e061 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:12.968747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8a263-8b3f-46fb-8141-4dc239c1e061 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:12.969457 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8a263-8b3f-46fb-8141-4dc239c1e061 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:12.969762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8a263-8b3f-46fb-8141-4dc239c1e061 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:12.971119 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8a263-8b3f-46fb-8141-4dc239c1e061 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:12.976738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 423/844] 10.222.0.67 () {56 vars in 1126 bytes} [Fri Jun 24 15:01:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:12.978299 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a58d2-c2a9-4b01-91e8-ae30deb4f804 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:12.978587 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a58d2-c2a9-4b01-91e8-ae30deb4f804 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:12.978863 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a58d2-c2a9-4b01-91e8-ae30deb4f804 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:12.980251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 424/845] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:12.984096 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a58d2-c2a9-4b01-91e8-ae30deb4f804 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:12.999821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096af961-1e3b-4441-8c8d-1301f6b118f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:13.000412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096af961-1e3b-4441-8c8d-1301f6b118f1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:13.000698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096af961-1e3b-4441-8c8d-1301f6b118f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:13.000976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096af961-1e3b-4441-8c8d-1301f6b118f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:13.070883 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096af961-1e3b-4441-8c8d-1301f6b118f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:13.072454 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096af961-1e3b-4441-8c8d-1301f6b118f1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:13.077789 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:13.096786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a58d2-c2a9-4b01-91e8-ae30deb4f804 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:13.130710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081aa149-92e0-4d11-961c-bb35b4a9eba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:13.130963 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:13.131640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081aa149-92e0-4d11-961c-bb35b4a9eba5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:13.131780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081aa149-92e0-4d11-961c-bb35b4a9eba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:13.131943 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081aa149-92e0-4d11-961c-bb35b4a9eba5 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:13.132650 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081aa149-92e0-4d11-961c-bb35b4a9eba5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:13.132956 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081aa149-92e0-4d11-961c-bb35b4a9eba5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:13.134319 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081aa149-92e0-4d11-961c-bb35b4a9eba5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:13.138598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 425/846] 10.222.0.67 () {56 vars in 1126 bytes} [Fri Jun 24 15:01:12 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:13.139808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b90b1-2cb9-4bf9-b67c-15ed0dbe5fd3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:13.140092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b90b1-2cb9-4bf9-b67c-15ed0dbe5fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:13.140360 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b90b1-2cb9-4bf9-b67c-15ed0dbe5fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:13.168336 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d99b90b1-2cb9-4bf9-b67c-15ed0dbe5fd3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:13.170284 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d99b90b1-2cb9-4bf9-b67c-15ed0dbe5fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:13.181649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 422/847] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:01:12 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:01:13.182824 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e84fd7-e55a-4272-9ba5-e5ca272fea9e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:13.183117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e84fd7-e55a-4272-9ba5-e5ca272fea9e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:13.183425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e84fd7-e55a-4272-9ba5-e5ca272fea9e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:13.184936 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 423/848] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:13.187289 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e84fd7-e55a-4272-9ba5-e5ca272fea9e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:13.253043 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 426/849] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:13.255336 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981c07f-d8db-41d3-adfe-2a46bad79d17 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:13.255820 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981c07f-d8db-41d3-adfe-2a46bad79d17 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:13.256676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981c07f-d8db-41d3-adfe-2a46bad79d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:13.282043 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9981c07f-d8db-41d3-adfe-2a46bad79d17 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:13.283268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9981c07f-d8db-41d3-adfe-2a46bad79d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:13.305526 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e84fd7-e55a-4272-9ba5-e5ca272fea9e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:13.342977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753ed81-7445-486a-976b-3b092d2fc9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:13.343917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753ed81-7445-486a-976b-3b092d2fc9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:13.344204 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753ed81-7445-486a-976b-3b092d2fc9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:13.344485 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753ed81-7445-486a-976b-3b092d2fc9f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:13.355979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753ed81-7445-486a-976b-3b092d2fc9f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:13.357600 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753ed81-7445-486a-976b-3b092d2fc9f6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:13.363400 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 427/850] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:13.364522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981c07f-d8db-41d3-adfe-2a46bad79d17 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:13.376236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:13.384687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-0753ed81-7445-486a-976b-3b092d2fc9f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:13.384687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:13.384687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:13.384687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:13.384687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:13.384687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:13.384687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:13.384687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:13.384687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:13.384687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:13.384687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:13.384687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:13.384687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:13.384687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:13.384687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:13.385637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:13.386534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:13.386534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:13.386534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:13.386534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:13.390903 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 424/851] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:13.391796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753ed81-7445-486a-976b-3b092d2fc9f6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:13.404807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:13.473634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981c07f-d8db-41d3-adfe-2a46bad79d17 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:13.498105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753ed81-7445-486a-976b-3b092d2fc9f6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:13.509327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a45df-1372-45a6-be06-d045af949c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:13.510061 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a45df-1372-45a6-be06-d045af949c13 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:13.510340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a45df-1372-45a6-be06-d045af949c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:13.510612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a45df-1372-45a6-be06-d045af949c13 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:13.511291 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a45df-1372-45a6-be06-d045af949c13 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:13.512650 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a45df-1372-45a6-be06-d045af949c13 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a06a45df-1372-45a6-be06-d045af949c13 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:13.524720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:13.525949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:13.526808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:13.526808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:13.526808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:13.526808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:13.526808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:13.526808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:13.526808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:13.526808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:13.526808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:13.526808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:13.526808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:13.526808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:13.526808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 15:01:13.526808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:13.526808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 428/852] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:01:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:13.527888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a45df-1372-45a6-be06-d045af949c13 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:13.543830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cfc1e-9827-4aa8-bcc0-7e5e1b022576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:13.544634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cfc1e-9827-4aa8-bcc0-7e5e1b022576 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:13.544920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cfc1e-9827-4aa8-bcc0-7e5e1b022576 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:13.545198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cfc1e-9827-4aa8-bcc0-7e5e1b022576 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:13.556578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27cfc1e-9827-4aa8-bcc0-7e5e1b022576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d27cfc1e-9827-4aa8-bcc0-7e5e1b022576 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:01:13.564198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:13.565156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:13.566927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:13.566927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:13.566927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:13.566927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:13.566927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:01:13.566927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:01:13.566927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:01:13.566927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:01:13.566927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:01:13.566927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:01:13.566927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:01:13.566927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:13.566927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 425/853] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:01:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:13.566927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cfc1e-9827-4aa8-bcc0-7e5e1b022576 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:13.613636 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:13.638338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a45df-1372-45a6-be06-d045af949c13 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:13.640616 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:13.674065 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40e412-2228-4053-9376-c6cf95e0730c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:13.674782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40e412-2228-4053-9376-c6cf95e0730c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:13.675111 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40e412-2228-4053-9376-c6cf95e0730c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:13.675426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40e412-2228-4053-9376-c6cf95e0730c None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:13.676157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40e412-2228-4053-9376-c6cf95e0730c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:13.676470 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40e412-2228-4053-9376-c6cf95e0730c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:13.678095 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40e412-2228-4053-9376-c6cf95e0730c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:13.682149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cfc1e-9827-4aa8-bcc0-7e5e1b022576 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:13.689785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 429/854] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:13.691113 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40e412-2228-4053-9376-c6cf95e0730c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:13.724142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfbcd0-33d6-4d20-aaaf-d21ac791b370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:13.725191 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfbcd0-33d6-4d20-aaaf-d21ac791b370 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:13.725685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfbcd0-33d6-4d20-aaaf-d21ac791b370 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:13.726133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfbcd0-33d6-4d20-aaaf-d21ac791b370 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:13.726970 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfbcd0-33d6-4d20-aaaf-d21ac791b370 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:13.727467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfbcd0-33d6-4d20-aaaf-d21ac791b370 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:13.729012 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfbcd0-33d6-4d20-aaaf-d21ac791b370 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:13.738835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 426/855] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:01:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:13.739911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfbcd0-33d6-4d20-aaaf-d21ac791b370 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:13.801205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40e412-2228-4053-9376-c6cf95e0730c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:13.837404 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165291c-0808-46e7-a5fe-c93b5a547fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:13.837995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165291c-0808-46e7-a5fe-c93b5a547fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:13.838280 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165291c-0808-46e7-a5fe-c93b5a547fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:13.838557 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165291c-0808-46e7-a5fe-c93b5a547fd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:13.839336 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165291c-0808-46e7-a5fe-c93b5a547fd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:13.839643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165291c-0808-46e7-a5fe-c93b5a547fd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:13.841005 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165291c-0808-46e7-a5fe-c93b5a547fd4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:13.842559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:13.848758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfbcd0-33d6-4d20-aaaf-d21ac791b370 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:13.850672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 430/856] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:01:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:13.851397 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165291c-0808-46e7-a5fe-c93b5a547fd4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:13.885937 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:13.890609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1738b-cce9-481c-a799-19dae48f826d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:13.891440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1738b-cce9-481c-a799-19dae48f826d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:13.891721 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1738b-cce9-481c-a799-19dae48f826d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:13.891994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1738b-cce9-481c-a799-19dae48f826d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:13.898944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1738b-cce9-481c-a799-19dae48f826d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:13.900408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1738b-cce9-481c-a799-19dae48f826d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:13.915305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-16c1738b-cce9-481c-a799-19dae48f826d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:13.915305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:13.915305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:13.915305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:13.915305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:13.915305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:13.915305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:13.915305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:13.915305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:13.915305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:13.915305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:13.915305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:13.915305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:01:13.915305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:01:13.915305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:13.916264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:13.917194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:01:13.917194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:01:13.917194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:01:13.917194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:01:13.917194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:01:13.917194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:01:13.917194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:13.917194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:13.917194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 427/857] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:13.917684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1738b-cce9-481c-a799-19dae48f826d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:13.962417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165291c-0808-46e7-a5fe-c93b5a547fd4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:13.999860 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb95b1-ee62-497e-b30f-a8b3c81b1d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:14.000516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb95b1-ee62-497e-b30f-a8b3c81b1d7a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:14.000796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb95b1-ee62-497e-b30f-a8b3c81b1d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:14.001070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb95b1-ee62-497e-b30f-a8b3c81b1d7a None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:14.001771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb95b1-ee62-497e-b30f-a8b3c81b1d7a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:14.002076 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb95b1-ee62-497e-b30f-a8b3c81b1d7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:14.003598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb95b1-ee62-497e-b30f-a8b3c81b1d7a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:14.008350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 431/858] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:01:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:14.017055 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb95b1-ee62-497e-b30f-a8b3c81b1d7a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:14.031715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1738b-cce9-481c-a799-19dae48f826d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:14.081700 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea4667-d578-4c7c-9e2c-0b1f768763a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:14.082923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea4667-d578-4c7c-9e2c-0b1f768763a2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:14.083734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea4667-d578-4c7c-9e2c-0b1f768763a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:14.084239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea4667-d578-4c7c-9e2c-0b1f768763a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:14.106208 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea4667-d578-4c7c-9e2c-0b1f768763a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:14.107732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea4667-d578-4c7c-9e2c-0b1f768763a2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:14.108319 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:14.128236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-80ea4667-d578-4c7c-9e2c-0b1f768763a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:14.128236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:14.128236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:14.128236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:14.128236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:14.128236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:14.128236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:14.128236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:14.128236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:14.128236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:14.128236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:14.128236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:14.128236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:14.128236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:14.128236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:14.129515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:14.130671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:14.130671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:14.130671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:14.130671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:14.135584 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 428/859] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:14.138109 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea4667-d578-4c7c-9e2c-0b1f768763a2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:14.169373 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb95b1-ee62-497e-b30f-a8b3c81b1d7a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:14.192342 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:14.208486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7296b37-40eb-48b7-b117-b6ccc8699eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:14.209163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7296b37-40eb-48b7-b117-b6ccc8699eac None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:14.209449 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7296b37-40eb-48b7-b117-b6ccc8699eac None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:14.209727 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7296b37-40eb-48b7-b117-b6ccc8699eac None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:14.210520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7296b37-40eb-48b7-b117-b6ccc8699eac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:14.210835 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7296b37-40eb-48b7-b117-b6ccc8699eac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:14.212314 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7296b37-40eb-48b7-b117-b6ccc8699eac None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:14.225661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 432/860] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:01:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:14.232970 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7296b37-40eb-48b7-b117-b6ccc8699eac None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:14.263836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea4667-d578-4c7c-9e2c-0b1f768763a2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:14.303629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceda24c-a89b-415d-97fa-94759ac9e0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:14.304405 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceda24c-a89b-415d-97fa-94759ac9e0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:14.304769 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceda24c-a89b-415d-97fa-94759ac9e0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:14.305119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceda24c-a89b-415d-97fa-94759ac9e0d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:14.305918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceda24c-a89b-415d-97fa-94759ac9e0d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:14.306292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceda24c-a89b-415d-97fa-94759ac9e0d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:14.307783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceda24c-a89b-415d-97fa-94759ac9e0d2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:14.324759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 429/861] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:14.325821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceda24c-a89b-415d-97fa-94759ac9e0d2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:14.353322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7296b37-40eb-48b7-b117-b6ccc8699eac None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:14.375443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:14.391592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70634653-ee40-4af0-90b7-808ca1d1f1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:14.394343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70634653-ee40-4af0-90b7-808ca1d1f1ce None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:14.394343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70634653-ee40-4af0-90b7-808ca1d1f1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:14.394343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70634653-ee40-4af0-90b7-808ca1d1f1ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:14.395106 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70634653-ee40-4af0-90b7-808ca1d1f1ce None admin] RBAC: Adding query filter params (role.id=716cc96fb28146b7b9ae6271569a206f, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=72f00ac4c6a247e485ac52a8dee4f1cb) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:14.395687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70634653-ee40-4af0-90b7-808ca1d1f1ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:14.397368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70634653-ee40-4af0-90b7-808ca1d1f1ce None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:14.404013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 433/862] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:01:14 2022] GET /identity/v3/role_assignments?user.id=72f00ac4c6a247e485ac52a8dee4f1cb&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=716cc96fb28146b7b9ae6271569a206f => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:14.442787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceda24c-a89b-415d-97fa-94759ac9e0d2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:14.458272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:14.481719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f92015-2231-4ca7-9c32-b6a25cdee5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:14.482552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f92015-2231-4ca7-9c32-b6a25cdee5cc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:14.483302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f92015-2231-4ca7-9c32-b6a25cdee5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:14.483985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f92015-2231-4ca7-9c32-b6a25cdee5cc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:14.493389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f92015-2231-4ca7-9c32-b6a25cdee5cc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:14.494896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f92015-2231-4ca7-9c32-b6a25cdee5cc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d9f92015-2231-4ca7-9c32-b6a25cdee5cc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:01:14.510755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:14.511858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:14.512829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:14.512829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:14.512829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:01:14.512829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:01:14.512829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:01:14.512829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:01:14.512829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:01:14.512829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:01:14.512829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 15:01:14.512829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:14.512829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 430/863] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:01:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:14.518019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70634653-ee40-4af0-90b7-808ca1d1f1ce None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:14.626208 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70634653-ee40-4af0-90b7-808ca1d1f1ce None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:14.661445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072124c-95c6-4892-b8af-9b0e91a00957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:14.662125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072124c-95c6-4892-b8af-9b0e91a00957 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:14.662411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072124c-95c6-4892-b8af-9b0e91a00957 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:14.662691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072124c-95c6-4892-b8af-9b0e91a00957 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:14.663568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072124c-95c6-4892-b8af-9b0e91a00957 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:14.663885 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072124c-95c6-4892-b8af-9b0e91a00957 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:14.665327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072124c-95c6-4892-b8af-9b0e91a00957 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:14.685148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 434/864] 10.222.0.67 () {56 vars in 1135 bytes} [Fri Jun 24 15:01:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:14.691605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f92015-2231-4ca7-9c32-b6a25cdee5cc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:14.798968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f92015-2231-4ca7-9c32-b6a25cdee5cc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:14.835410 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ee8e5-6595-4525-9534-ab182f651fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:14.836070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ee8e5-6595-4525-9534-ab182f651fbd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:14.836354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ee8e5-6595-4525-9534-ab182f651fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:14.836633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ee8e5-6595-4525-9534-ab182f651fbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:14.873768 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:14.920280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ee8e5-6595-4525-9534-ab182f651fbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:14.922082 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ee8e5-6595-4525-9534-ab182f651fbd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:15.030759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 431/865] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:01:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:01:15.036806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072124c-95c6-4892-b8af-9b0e91a00957 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:15.092913 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:15.154771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072124c-95c6-4892-b8af-9b0e91a00957 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:15.192739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf477a56-8375-4dad-960b-495afa15f783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:15.193518 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf477a56-8375-4dad-960b-495afa15f783 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:15.193804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf477a56-8375-4dad-960b-495afa15f783 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:15.194131 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf477a56-8375-4dad-960b-495afa15f783 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:15.204427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf477a56-8375-4dad-960b-495afa15f783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:15.205891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf477a56-8375-4dad-960b-495afa15f783 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:15.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-bf477a56-8375-4dad-960b-495afa15f783 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:15.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:15.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:15.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:15.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:15.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:15.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:15.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:15.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:15.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:15.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:15.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:15.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:15.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:15.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:15.225669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:15.226568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:15.226568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:15.226568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:15.226568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:15.226568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 435/866] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:15.231798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ee8e5-6595-4525-9534-ab182f651fbd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:15.236739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42482e70-ad5c-4293-90e4-224c4ad541cc None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:15.237026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42482e70-ad5c-4293-90e4-224c4ad541cc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:15.237305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42482e70-ad5c-4293-90e4-224c4ad541cc None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:15.239633 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 436/867] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:15.244927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba7fe6-f6ea-49f2-94aa-03d5eb3cd12b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:15.245219 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba7fe6-f6ea-49f2-94aa-03d5eb3cd12b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:15.245500 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba7fe6-f6ea-49f2-94aa-03d5eb3cd12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:15.263750 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a6ba7fe6-f6ea-49f2-94aa-03d5eb3cd12b None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:15.265025 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a6ba7fe6-f6ea-49f2-94aa-03d5eb3cd12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:15.302815 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:15.349491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ee8e5-6595-4525-9534-ab182f651fbd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:15.363659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 437/868] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:15.380896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba7fe6-f6ea-49f2-94aa-03d5eb3cd12b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:15.401773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25facab-553d-4cc0-bba0-6094040a2b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:15.402718 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25facab-553d-4cc0-bba0-6094040a2b22 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:15.403230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25facab-553d-4cc0-bba0-6094040a2b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:15.404265 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25facab-553d-4cc0-bba0-6094040a2b22 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:15.405468 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25facab-553d-4cc0-bba0-6094040a2b22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:15.405977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25facab-553d-4cc0-bba0-6094040a2b22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:15.407658 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:15.410258 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25facab-553d-4cc0-bba0-6094040a2b22 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:15.430375 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 432/869] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:15.436129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25facab-553d-4cc0-bba0-6094040a2b22 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:15.501357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba7fe6-f6ea-49f2-94aa-03d5eb3cd12b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:15.547523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157148d4-ab3c-4d11-a380-fe2c01529e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:15.548125 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25facab-553d-4cc0-bba0-6094040a2b22 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:15.548716 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157148d4-ab3c-4d11-a380-fe2c01529e27 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:15.549176 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157148d4-ab3c-4d11-a380-fe2c01529e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:15.549645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157148d4-ab3c-4d11-a380-fe2c01529e27 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:15.552123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157148d4-ab3c-4d11-a380-fe2c01529e27 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:15.553695 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157148d4-ab3c-4d11-a380-fe2c01529e27 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:15.574849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:15.586486 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3a8bb-4974-4492-b1d1-fc0cc2f83eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:15.587338 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3a8bb-4974-4492-b1d1-fc0cc2f83eba None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:15.587631 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3a8bb-4974-4492-b1d1-fc0cc2f83eba None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:15.587906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3a8bb-4974-4492-b1d1-fc0cc2f83eba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:15.594642 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3a8bb-4974-4492-b1d1-fc0cc2f83eba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:15.596107 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3a8bb-4974-4492-b1d1-fc0cc2f83eba None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:15.610840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2ff3a8bb-4974-4492-b1d1-fc0cc2f83eba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:15.610840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:15.610840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:15.610840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:15.610840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:15.610840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:15.610840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:15.610840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:15.610840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:15.610840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:15.610840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:15.610840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:15.610840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:01:15.610840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:01:15.610840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:15.612041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:15.613030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:01:15.613030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:01:15.613030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:01:15.613030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:01:15.613030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:01:15.613030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:01:15.613030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:15.613030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:15.615500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 433/870] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:15.617795 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3a8bb-4974-4492-b1d1-fc0cc2f83eba None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:15.643503 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 438/871] 10.222.0.67 () {60 vars in 1141 bytes} [Fri Jun 24 15:01:15 2022] POST /identity/v3/services => generated 269 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:01:15.749793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3a8bb-4974-4492-b1d1-fc0cc2f83eba None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:15.797037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf2eba-e15a-48c5-b9b7-74f0423a41f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:15.798102 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf2eba-e15a-48c5-b9b7-74f0423a41f2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:15.798612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf2eba-e15a-48c5-b9b7-74f0423a41f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:15.799104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf2eba-e15a-48c5-b9b7-74f0423a41f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:15.800050 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf2eba-e15a-48c5-b9b7-74f0423a41f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:15.800994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf2eba-e15a-48c5-b9b7-74f0423a41f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:15.802718 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf2eba-e15a-48c5-b9b7-74f0423a41f2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:15.816563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 434/872] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:01:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:15.822641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157148d4-ab3c-4d11-a380-fe2c01529e27 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:15.914827 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:15.931846 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157148d4-ab3c-4d11-a380-fe2c01529e27 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:15.968123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544024df-b8f7-4bd5-bcf3-f3921ec3648f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:15.968799 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544024df-b8f7-4bd5-bcf3-f3921ec3648f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:15.969086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544024df-b8f7-4bd5-bcf3-f3921ec3648f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:15.969364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544024df-b8f7-4bd5-bcf3-f3921ec3648f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:15.970200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544024df-b8f7-4bd5-bcf3-f3921ec3648f None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=e4732ab3c8c446958092246d16b4b4f0) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:15.970516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544024df-b8f7-4bd5-bcf3-f3921ec3648f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:15.972656 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544024df-b8f7-4bd5-bcf3-f3921ec3648f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:15.979420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 439/873] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:01:15 2022] GET /identity/v3/role_assignments?user.id=e4732ab3c8c446958092246d16b4b4f0&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:16.122538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48375a4-65fe-4893-aa86-073111fd3434 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:16.122884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48375a4-65fe-4893-aa86-073111fd3434 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:16.123103 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48375a4-65fe-4893-aa86-073111fd3434 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:16.124887 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 435/874] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:16.131495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fca265-5816-4052-ae56-e3da865946f4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:16.132010 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fca265-5816-4052-ae56-e3da865946f4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:16.132089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fca265-5816-4052-ae56-e3da865946f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:16.148908 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f9fca265-5816-4052-ae56-e3da865946f4 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:16.150088 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f9fca265-5816-4052-ae56-e3da865946f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:16.224157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 440/875] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:16 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:16.246870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48375a4-65fe-4893-aa86-073111fd3434 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:16.292146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:16.351552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48375a4-65fe-4893-aa86-073111fd3434 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:16.389110 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5314f-7bc2-4303-b81a-cf44a05c00de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:16.389795 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5314f-7bc2-4303-b81a-cf44a05c00de None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:16.390088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5314f-7bc2-4303-b81a-cf44a05c00de None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:16.390375 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5314f-7bc2-4303-b81a-cf44a05c00de None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:16.391120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5314f-7bc2-4303-b81a-cf44a05c00de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:16.392983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5314f-7bc2-4303-b81a-cf44a05c00de None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:16.468239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 436/876] 10.222.0.67 () {60 vars in 1141 bytes} [Fri Jun 24 15:01:16 2022] POST /identity/v3/services => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:01:16.685409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb2b07-4d58-41b3-8b11-9d501be0da13 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:16.685713 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb2b07-4d58-41b3-8b11-9d501be0da13 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:16.685970 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb2b07-4d58-41b3-8b11-9d501be0da13 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:16.687642 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 441/877] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:16.693611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c986d-deeb-46d3-9599-449d4720ba54 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:16.693899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c986d-deeb-46d3-9599-449d4720ba54 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:16.694175 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c986d-deeb-46d3-9599-449d4720ba54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:16.711366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-027c986d-deeb-46d3-9599-449d4720ba54 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:16.712549 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-027c986d-deeb-46d3-9599-449d4720ba54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:16.793645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 437/878] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:16.818185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb2b07-4d58-41b3-8b11-9d501be0da13 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:16.906619 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:16.930307 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb2b07-4d58-41b3-8b11-9d501be0da13 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:16.969386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e39eb6-22ca-47e9-8349-4263f4303cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:16.969386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e39eb6-22ca-47e9-8349-4263f4303cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:16.969386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e39eb6-22ca-47e9-8349-4263f4303cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:16.969386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e39eb6-22ca-47e9-8349-4263f4303cb0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:16.980417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e39eb6-22ca-47e9-8349-4263f4303cb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-46e39eb6-22ca-47e9-8349-4263f4303cb0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:01:16.989507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:16.990672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:16.991784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:16.991784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:16.991784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:16.991784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:16.991784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:01:16.991784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:01:16.991784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:01:16.991784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:01:16.991784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:01:16.991784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:01:16.991784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 15:01:16.991784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:16.994193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 442/879] 10.222.0.67 () {56 vars in 1120 bytes} [Fri Jun 24 15:01:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:16.999756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c986d-deeb-46d3-9599-449d4720ba54 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:17.027550 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:17.101246 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c986d-deeb-46d3-9599-449d4720ba54 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:17.135811 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed9a50-92bf-4228-a697-b59922d19bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:17.136484 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed9a50-92bf-4228-a697-b59922d19bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:17.136766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed9a50-92bf-4228-a697-b59922d19bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:17.137040 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed9a50-92bf-4228-a697-b59922d19bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:17.137782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed9a50-92bf-4228-a697-b59922d19bf7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:17.138083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed9a50-92bf-4228-a697-b59922d19bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:17.139458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed9a50-92bf-4228-a697-b59922d19bf7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:17.148598 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 438/880] 10.222.0.67 () {56 vars in 1115 bytes} [Fri Jun 24 15:01:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:17.154000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e39eb6-22ca-47e9-8349-4263f4303cb0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:17.264217 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e39eb6-22ca-47e9-8349-4263f4303cb0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:17.299622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03ea7f-faef-4c9d-8120-140c6bb93ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:17.301140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03ea7f-faef-4c9d-8120-140c6bb93ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:17.301420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03ea7f-faef-4c9d-8120-140c6bb93ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:17.301697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03ea7f-faef-4c9d-8120-140c6bb93ae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:17.309048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:17.316063 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03ea7f-faef-4c9d-8120-140c6bb93ae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:17.317371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03ea7f-faef-4c9d-8120-140c6bb93ae6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:17.334831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ef03ea7f-faef-4c9d-8120-140c6bb93ae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:17.334831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:17.334831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:17.334831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:17.334831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:17.334831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:17.334831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:17.334831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:17.334831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:17.334831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:17.334831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:17.334831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:17.334831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:17.334831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:17.334831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:17.335831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:17.337327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:17.337327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:17.337327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:17.337327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:17.337327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 443/881] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:17.341731 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed9a50-92bf-4228-a697-b59922d19bf7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:17.436418 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:17.448398 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed9a50-92bf-4228-a697-b59922d19bf7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:17.482669 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84e40b-46bb-4450-ad2a-b8cbc9a88dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:17.483364 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84e40b-46bb-4450-ad2a-b8cbc9a88dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:17.483981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84e40b-46bb-4450-ad2a-b8cbc9a88dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:17.484469 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84e40b-46bb-4450-ad2a-b8cbc9a88dd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:17.485709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84e40b-46bb-4450-ad2a-b8cbc9a88dd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:17.486017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84e40b-46bb-4450-ad2a-b8cbc9a88dd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:17.488254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84e40b-46bb-4450-ad2a-b8cbc9a88dd2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:17.500333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 439/882] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:17.505777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03ea7f-faef-4c9d-8120-140c6bb93ae6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:17.614867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03ea7f-faef-4c9d-8120-140c6bb93ae6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:17.654337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96461b-2a46-440c-b9e6-16c104aa1828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:17.655159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96461b-2a46-440c-b9e6-16c104aa1828 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:17.655470 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96461b-2a46-440c-b9e6-16c104aa1828 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:17.655747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96461b-2a46-440c-b9e6-16c104aa1828 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:17.664159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96461b-2a46-440c-b9e6-16c104aa1828 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:17.665612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96461b-2a46-440c-b9e6-16c104aa1828 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ef96461b-2a46-440c-b9e6-16c104aa1828 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:01:17.682193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:17.683269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:17.684236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:17.684236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:17.684236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:01:17.684236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:01:17.684236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:01:17.684236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:01:17.684236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:01:17.684236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:01:17.684236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 15:01:17.684236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:17.684236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 444/883] 10.222.0.67 () {56 vars in 1117 bytes} [Fri Jun 24 15:01:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:17.693070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84e40b-46bb-4450-ad2a-b8cbc9a88dd2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:17.796794 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84e40b-46bb-4450-ad2a-b8cbc9a88dd2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:17.834949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20611402-f806-4ca7-85dc-a215b6027ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:17.835658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20611402-f806-4ca7-85dc-a215b6027ead None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:17.835938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20611402-f806-4ca7-85dc-a215b6027ead None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:17.836215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20611402-f806-4ca7-85dc-a215b6027ead None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:17.837050 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20611402-f806-4ca7-85dc-a215b6027ead None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:17.837355 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20611402-f806-4ca7-85dc-a215b6027ead None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:17.838767 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20611402-f806-4ca7-85dc-a215b6027ead None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:17.856920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 440/884] 10.222.0.67 () {56 vars in 1149 bytes} [Fri Jun 24 15:01:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:17.863263 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96461b-2a46-440c-b9e6-16c104aa1828 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:17.871787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e5586-a326-4106-aec8-368d6a710eec None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:17.872025 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e5586-a326-4106-aec8-368d6a710eec None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:17.872276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e5586-a326-4106-aec8-368d6a710eec None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:17.873856 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 441/885] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:01:17.879741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594ddcd-142b-4ad9-81ae-9e1166d90a21 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:17.880040 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594ddcd-142b-4ad9-81ae-9e1166d90a21 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:17.880337 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594ddcd-142b-4ad9-81ae-9e1166d90a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:17.893754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:17.901065 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a594ddcd-142b-4ad9-81ae-9e1166d90a21 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:17.902439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a594ddcd-142b-4ad9-81ae-9e1166d90a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:17.919594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:17.984238 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96461b-2a46-440c-b9e6-16c104aa1828 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:17.985564 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 442/886] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:18.009598 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594ddcd-142b-4ad9-81ae-9e1166d90a21 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:18.030315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e4cad-8d19-4056-ac04-436d5a615979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:18.031902 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e4cad-8d19-4056-ac04-436d5a615979 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:18.032414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e4cad-8d19-4056-ac04-436d5a615979 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:18.033147 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e4cad-8d19-4056-ac04-436d5a615979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:18.132074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e4cad-8d19-4056-ac04-436d5a615979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:18.133436 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e4cad-8d19-4056-ac04-436d5a615979 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:18.134484 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594ddcd-142b-4ad9-81ae-9e1166d90a21 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:18.181213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdcd76-6503-4730-926e-b9410df4a3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:18.182048 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdcd76-6503-4730-926e-b9410df4a3eb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:18.182336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdcd76-6503-4730-926e-b9410df4a3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:18.182617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdcd76-6503-4730-926e-b9410df4a3eb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:18.183310 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacdcd76-6503-4730-926e-b9410df4a3eb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:18.184687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacdcd76-6503-4730-926e-b9410df4a3eb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:18.187783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:18.194977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-dacdcd76-6503-4730-926e-b9410df4a3eb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:18.200906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:18.201828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:18.202677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:18.202677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:18.202677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:18.202677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:18.202677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:18.202677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:18.202677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:18.202677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:18.202677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:18.202677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:18.202677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:18.202677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:18.202677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:18.202677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 15:01:18.202677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:18.205023 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 443/887] 10.222.0.67 () {56 vars in 1132 bytes} [Fri Jun 24 15:01:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:18.206416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdcd76-6503-4730-926e-b9410df4a3eb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:18.261198 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 445/888] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:01:17 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:01:18.263103 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceef286-12a2-4a12-8e00-f6f3692f39a2 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:18.263439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceef286-12a2-4a12-8e00-f6f3692f39a2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:18.263714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceef286-12a2-4a12-8e00-f6f3692f39a2 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:18.265007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 446/889] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:18.265679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceef286-12a2-4a12-8e00-f6f3692f39a2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:18.323848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdcd76-6503-4730-926e-b9410df4a3eb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:18.359736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17215d-867e-4e5b-9daf-46d25a5fb70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:18.360395 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17215d-867e-4e5b-9daf-46d25a5fb70e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:18.360766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17215d-867e-4e5b-9daf-46d25a5fb70e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:18.360949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17215d-867e-4e5b-9daf-46d25a5fb70e None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:18.361640 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17215d-867e-4e5b-9daf-46d25a5fb70e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:18.361941 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17215d-867e-4e5b-9daf-46d25a5fb70e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:18.363403 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17215d-867e-4e5b-9daf-46d25a5fb70e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:18.368653 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 444/890] 10.222.0.67 () {56 vars in 1126 bytes} [Fri Jun 24 15:01:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:18.369644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befdcad-39b3-41a9-b757-c2d863b247b2 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:18.369929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befdcad-39b3-41a9-b757-c2d863b247b2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:18.370205 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befdcad-39b3-41a9-b757-c2d863b247b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:18.383619 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceef286-12a2-4a12-8e00-f6f3692f39a2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:18.386863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3befdcad-39b3-41a9-b757-c2d863b247b2 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:18.388081 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3befdcad-39b3-41a9-b757-c2d863b247b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:18.421634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cd53f-1e33-4dd8-b428-9c9d64e3f6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:18.422724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cd53f-1e33-4dd8-b428-9c9d64e3f6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:18.423159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cd53f-1e33-4dd8-b428-9c9d64e3f6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:18.423463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cd53f-1e33-4dd8-b428-9c9d64e3f6d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:18.434111 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cd53f-1e33-4dd8-b428-9c9d64e3f6d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:18.435567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cd53f-1e33-4dd8-b428-9c9d64e3f6d2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:18.452283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:18.453256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d25cd53f-1e33-4dd8-b428-9c9d64e3f6d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:18.453256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:18.453256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:18.453256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:18.453256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:18.453256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:18.453256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:18.453256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:18.453256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:18.453256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:18.453256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:18.453256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:18.453256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:18.453256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:18.453256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:18.454180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:18.455081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:18.455081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:18.455081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:18.455081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:18.455081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 447/891] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:18.455809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cd53f-1e33-4dd8-b428-9c9d64e3f6d2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:18.458567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:18.466661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 445/892] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:18.467752 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befdcad-39b3-41a9-b757-c2d863b247b2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:18.567916 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cd53f-1e33-4dd8-b428-9c9d64e3f6d2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:18.579372 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befdcad-39b3-41a9-b757-c2d863b247b2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:18.603774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428c718-d285-4446-9132-4aff298f75e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:18.604446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428c718-d285-4446-9132-4aff298f75e9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:18.604743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428c718-d285-4446-9132-4aff298f75e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:18.605011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428c718-d285-4446-9132-4aff298f75e9 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:18.605725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428c718-d285-4446-9132-4aff298f75e9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:18.606038 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428c718-d285-4446-9132-4aff298f75e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:18.607428 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428c718-d285-4446-9132-4aff298f75e9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:18.613309 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 448/893] 10.222.0.67 () {56 vars in 1126 bytes} [Fri Jun 24 15:01:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:18.617610 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52277b-c860-4d73-aa49-0e75f7f05651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:18.618516 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52277b-c860-4d73-aa49-0e75f7f05651 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:18.618754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428c718-d285-4446-9132-4aff298f75e9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:18.618854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52277b-c860-4d73-aa49-0e75f7f05651 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:18.619119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52277b-c860-4d73-aa49-0e75f7f05651 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:18.619885 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52277b-c860-4d73-aa49-0e75f7f05651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:18.620187 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52277b-c860-4d73-aa49-0e75f7f05651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:18.621540 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52277b-c860-4d73-aa49-0e75f7f05651 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:18.638118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 446/894] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:18.638118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52277b-c860-4d73-aa49-0e75f7f05651 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:18.688207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:18.692155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:18.723420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428c718-d285-4446-9132-4aff298f75e9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:18.734814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52277b-c860-4d73-aa49-0e75f7f05651 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:18.756314 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adefb3c-f09f-441c-a3ce-c099e931a99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:18.757121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adefb3c-f09f-441c-a3ce-c099e931a99d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:18.757412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adefb3c-f09f-441c-a3ce-c099e931a99d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:18.757687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adefb3c-f09f-441c-a3ce-c099e931a99d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:18.758341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adefb3c-f09f-441c-a3ce-c099e931a99d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:18.759775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adefb3c-f09f-441c-a3ce-c099e931a99d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:18.769587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d0333-e9e9-4774-b61a-0c1d7015c4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:18.774667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d0333-e9e9-4774-b61a-0c1d7015c4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:18.774667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d0333-e9e9-4774-b61a-0c1d7015c4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:18.774667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d0333-e9e9-4774-b61a-0c1d7015c4c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:18.774667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d0333-e9e9-4774-b61a-0c1d7015c4c7 None admin] RBAC: Adding query filter params (interface=public, service_id=855ecbd5ce804764b8c5d5dd78e31037, region_id=RegionOne) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:18.774667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d0333-e9e9-4774-b61a-0c1d7015c4c7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-3adefb3c-f09f-441c-a3ce-c099e931a99d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:18.775064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:18.776233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:18.777204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:18.777204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:18.777204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:18.777204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:18.777204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:18.777204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:18.777204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:18.777204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:18.777204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:18.777204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:18.777204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:18.777204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:18.777204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 15:01:18.777204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:18.778326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d0333-e9e9-4774-b61a-0c1d7015c4c7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:18.778409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 449/895] 10.222.0.67 () {56 vars in 1126 bytes} [Fri Jun 24 15:01:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:18.778409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ab7a1-c5a6-4b12-990a-58f1d2e042de None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:18.778409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ab7a1-c5a6-4b12-990a-58f1d2e042de None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:18.778409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ab7a1-c5a6-4b12-990a-58f1d2e042de None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:18.778409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 450/896] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:18.778409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ab7a1-c5a6-4b12-990a-58f1d2e042de None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:18.783283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 447/897] 10.222.0.67 () {56 vars in 1251 bytes} [Fri Jun 24 15:01:18 2022] GET /identity/v3/endpoints?service_id=855ecbd5ce804764b8c5d5dd78e31037&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:18.784839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d0333-e9e9-4774-b61a-0c1d7015c4c7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:18.879608 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ab7a1-c5a6-4b12-990a-58f1d2e042de None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:18.884360 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d0333-e9e9-4774-b61a-0c1d7015c4c7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:18.900949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:18.907835 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:18.914101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd496e-4fcc-44ac-8300-6db8885715d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:18.914878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd496e-4fcc-44ac-8300-6db8885715d1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:18.915158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd496e-4fcc-44ac-8300-6db8885715d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:18.915460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd496e-4fcc-44ac-8300-6db8885715d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:18.917727 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf02ac-0671-4032-ac0d-81a10e3f5af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:18.918385 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf02ac-0671-4032-ac0d-81a10e3f5af7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:18.918671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf02ac-0671-4032-ac0d-81a10e3f5af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:18.918946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf02ac-0671-4032-ac0d-81a10e3f5af7 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:18.919747 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf02ac-0671-4032-ac0d-81a10e3f5af7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:18.920056 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf02ac-0671-4032-ac0d-81a10e3f5af7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:18.921409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf02ac-0671-4032-ac0d-81a10e3f5af7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:18.922005 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd496e-4fcc-44ac-8300-6db8885715d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:18.923443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd496e-4fcc-44ac-8300-6db8885715d1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:18.927066 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 448/898] 10.222.0.67 () {56 vars in 1122 bytes} [Fri Jun 24 15:01:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:18.928193 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec308534-0e65-4b37-a424-c932b1b0cfe8 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:18.928475 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec308534-0e65-4b37-a424-c932b1b0cfe8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:18.928748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec308534-0e65-4b37-a424-c932b1b0cfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:18.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-dfbd496e-4fcc-44ac-8300-6db8885715d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:18.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:18.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:18.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:18.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:18.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:18.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:18.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:18.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:18.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:18.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:18.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:18.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:01:18.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:01:18.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:18.939383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:18.940281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:01:18.940281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:01:18.940281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:01:18.940281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:01:18.940281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:01:18.940281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:01:18.940281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:18.940281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:18.940281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 451/899] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:18.941360 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd496e-4fcc-44ac-8300-6db8885715d1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:18.946775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ec308534-0e65-4b37-a424-c932b1b0cfe8 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:18.947973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ec308534-0e65-4b37-a424-c932b1b0cfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:19.014389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 449/900] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:19.015345 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec308534-0e65-4b37-a424-c932b1b0cfe8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:19.041165 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd496e-4fcc-44ac-8300-6db8885715d1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:19.075364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1b02f-65a5-48e6-98e7-81099e2dce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:19.076264 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1b02f-65a5-48e6-98e7-81099e2dce9c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:19.076554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1b02f-65a5-48e6-98e7-81099e2dce9c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:19.076832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1b02f-65a5-48e6-98e7-81099e2dce9c None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:19.077517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1b02f-65a5-48e6-98e7-81099e2dce9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:19.078874 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1b02f-65a5-48e6-98e7-81099e2dce9c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:19.085139 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 452/901] 10.222.0.67 () {56 vars in 1087 bytes} [Fri Jun 24 15:01:18 2022] GET /identity/v3/services => generated 1268 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:01:19.085917 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1b02f-65a5-48e6-98e7-81099e2dce9c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:19.132207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec308534-0e65-4b37-a424-c932b1b0cfe8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:19.148057 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:19.169047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:19.172489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b31220-68b7-42ec-9df7-b0eb3ac12169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:19.173185 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b31220-68b7-42ec-9df7-b0eb3ac12169 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:19.173470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b31220-68b7-42ec-9df7-b0eb3ac12169 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:19.173750 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b31220-68b7-42ec-9df7-b0eb3ac12169 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:19.174467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b31220-68b7-42ec-9df7-b0eb3ac12169 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:19.174809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b31220-68b7-42ec-9df7-b0eb3ac12169 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:19.176559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b31220-68b7-42ec-9df7-b0eb3ac12169 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:19.181961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 450/902] 10.222.0.67 () {56 vars in 1122 bytes} [Fri Jun 24 15:01:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:19.182810 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b31220-68b7-42ec-9df7-b0eb3ac12169 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:19.217728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1b02f-65a5-48e6-98e7-81099e2dce9c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:19.263985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfd5c7-56c4-490e-8d25-7f79a492e4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:19.264689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfd5c7-56c4-490e-8d25-7f79a492e4af None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:19.264956 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfd5c7-56c4-490e-8d25-7f79a492e4af None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:19.265233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfd5c7-56c4-490e-8d25-7f79a492e4af None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:19.266044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfd5c7-56c4-490e-8d25-7f79a492e4af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:19.266358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfd5c7-56c4-490e-8d25-7f79a492e4af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:19.268214 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfd5c7-56c4-490e-8d25-7f79a492e4af None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:19.290305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 453/903] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:01:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:19.291603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfd5c7-56c4-490e-8d25-7f79a492e4af None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:19.349054 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b31220-68b7-42ec-9df7-b0eb3ac12169 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:19.398304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51926913-1633-476b-a958-754caa898ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:19.399642 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51926913-1633-476b-a958-754caa898ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:19.400357 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51926913-1633-476b-a958-754caa898ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:19.400778 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51926913-1633-476b-a958-754caa898ee2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:19.402112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51926913-1633-476b-a958-754caa898ee2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:19.404036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51926913-1633-476b-a958-754caa898ee2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:19.407156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfd5c7-56c4-490e-8d25-7f79a492e4af None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-51926913-1633-476b-a958-754caa898ee2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:19.419992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:19.421237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:19.422286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:19.422286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:19.422286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:19.422286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:19.422286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:19.422286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:19.422286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:19.422286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:19.422286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:19.422286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:19.422286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:19.422286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:19.422286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 15:01:19.422286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:19.425314 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:19.427210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 451/904] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:01:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:19.436787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51926913-1633-476b-a958-754caa898ee2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:19.446783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eda026-a5dc-43b3-8a59-4b5702dd0ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:19.447622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eda026-a5dc-43b3-8a59-4b5702dd0ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:19.447999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eda026-a5dc-43b3-8a59-4b5702dd0ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:19.448281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eda026-a5dc-43b3-8a59-4b5702dd0ab3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:19.449109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eda026-a5dc-43b3-8a59-4b5702dd0ab3 None admin] RBAC: Adding query filter params (role.id=f70de858079b47cf92a3995fda64bbc0, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=72f00ac4c6a247e485ac52a8dee4f1cb) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:19.449429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eda026-a5dc-43b3-8a59-4b5702dd0ab3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:19.450868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eda026-a5dc-43b3-8a59-4b5702dd0ab3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:19.457771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 454/905] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:01:19 2022] GET /identity/v3/role_assignments?user.id=72f00ac4c6a247e485ac52a8dee4f1cb&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=f70de858079b47cf92a3995fda64bbc0 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:19.507593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:19.567213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51926913-1633-476b-a958-754caa898ee2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:19.605267 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83085192-37de-4bfc-9b47-829e0b00d3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:19.606186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83085192-37de-4bfc-9b47-829e0b00d3df None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:19.606656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83085192-37de-4bfc-9b47-829e0b00d3df None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:19.606925 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83085192-37de-4bfc-9b47-829e0b00d3df None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:19.607756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83085192-37de-4bfc-9b47-829e0b00d3df None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:19.607970 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83085192-37de-4bfc-9b47-829e0b00d3df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:19.609341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83085192-37de-4bfc-9b47-829e0b00d3df None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:19.615092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 452/906] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:01:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:19.620690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eda026-a5dc-43b3-8a59-4b5702dd0ab3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:19.732286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eda026-a5dc-43b3-8a59-4b5702dd0ab3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:19.767781 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef869378-8b03-4646-8bbc-614e0a22fcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:19.768458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef869378-8b03-4646-8bbc-614e0a22fcb4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:19.769263 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef869378-8b03-4646-8bbc-614e0a22fcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:19.769536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef869378-8b03-4646-8bbc-614e0a22fcb4 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:19.770249 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef869378-8b03-4646-8bbc-614e0a22fcb4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:19.770557 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef869378-8b03-4646-8bbc-614e0a22fcb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:19.771957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef869378-8b03-4646-8bbc-614e0a22fcb4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:19.777627 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 455/907] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:01:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:19.783333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83085192-37de-4bfc-9b47-829e0b00d3df None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:19.889776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83085192-37de-4bfc-9b47-829e0b00d3df None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:19.892148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:19.924650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7b3aa-9df5-4247-aec4-cb83dd42094e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:19.926017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7b3aa-9df5-4247-aec4-cb83dd42094e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:19.926302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7b3aa-9df5-4247-aec4-cb83dd42094e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:19.926546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7b3aa-9df5-4247-aec4-cb83dd42094e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:19.927985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7b3aa-9df5-4247-aec4-cb83dd42094e None admin] RBAC: Adding query filter params (interface=public, service_id=bf57b95d197043b5adea761b0f488f57, region_id=RegionOne) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:19.928263 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7b3aa-9df5-4247-aec4-cb83dd42094e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:19.931239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7b3aa-9df5-4247-aec4-cb83dd42094e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:19.937097 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 453/908] 10.222.0.67 () {56 vars in 1251 bytes} [Fri Jun 24 15:01:19 2022] GET /identity/v3/endpoints?service_id=bf57b95d197043b5adea761b0f488f57&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:19.943211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef869378-8b03-4646-8bbc-614e0a22fcb4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:19.989969 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:20.047528 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef869378-8b03-4646-8bbc-614e0a22fcb4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:20.084257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b02d2-f12a-4e24-a6a5-a55bbc2c2b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:20.084918 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b02d2-f12a-4e24-a6a5-a55bbc2c2b94 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:20.085261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b02d2-f12a-4e24-a6a5-a55bbc2c2b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:20.085549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b02d2-f12a-4e24-a6a5-a55bbc2c2b94 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:20.091254 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b02d2-f12a-4e24-a6a5-a55bbc2c2b94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:20.091254 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b02d2-f12a-4e24-a6a5-a55bbc2c2b94 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:20.094091 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 456/909] 10.222.0.67 () {56 vars in 1087 bytes} [Fri Jun 24 15:01:19 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:01:21.243705 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d60a6-afa2-4320-b731-baac15e74f5b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:21.243705 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d60a6-afa2-4320-b731-baac15e74f5b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:21.243705 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d60a6-afa2-4320-b731-baac15e74f5b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:21.244654 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 454/910] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:21.249960 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3ca99-3505-4f93-96a8-933b01861f84 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:21.250699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3ca99-3505-4f93-96a8-933b01861f84 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:21.251120 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3ca99-3505-4f93-96a8-933b01861f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:21.269796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-55f3ca99-3505-4f93-96a8-933b01861f84 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:21.271199 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-55f3ca99-3505-4f93-96a8-933b01861f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:21.303266 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b52c9-f1df-4fe2-81c7-77de81720a8a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:21.303554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b52c9-f1df-4fe2-81c7-77de81720a8a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:21.304012 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b52c9-f1df-4fe2-81c7-77de81720a8a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:21.305380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 455/911] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:01:21.310576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096c557-bb6e-4e57-b8e5-701af0e94ba0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:21.311104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096c557-bb6e-4e57-b8e5-701af0e94ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:21.311636 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096c557-bb6e-4e57-b8e5-701af0e94ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:21.329929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2096c557-bb6e-4e57-b8e5-701af0e94ba0 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:21.331507 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2096c557-bb6e-4e57-b8e5-701af0e94ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:21.346115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 457/912] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:21.369183 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3ca99-3505-4f93-96a8-933b01861f84 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:21.404350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:21.419268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 456/913] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:21.444016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096c557-bb6e-4e57-b8e5-701af0e94ba0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:21.496367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3ca99-3505-4f93-96a8-933b01861f84 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:21.542406 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fac65-f1d1-43b8-8510-eb45b3adf6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:21.543626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fac65-f1d1-43b8-8510-eb45b3adf6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:21.544117 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fac65-f1d1-43b8-8510-eb45b3adf6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:21.544594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fac65-f1d1-43b8-8510-eb45b3adf6e8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:21.546003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443fac65-f1d1-43b8-8510-eb45b3adf6e8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:21.548210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443fac65-f1d1-43b8-8510-eb45b3adf6e8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:21.555544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:21.562821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096c557-bb6e-4e57-b8e5-701af0e94ba0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-443fac65-f1d1-43b8-8510-eb45b3adf6e8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:21.565211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:21.566330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:21.567416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 458/914] 10.222.0.67 () {56 vars in 1132 bytes} [Fri Jun 24 15:01:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:21.568472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e510a0-142e-400e-91a2-f251d58a55fb None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:21.568472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e510a0-142e-400e-91a2-f251d58a55fb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:21.568472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e510a0-142e-400e-91a2-f251d58a55fb None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:21.568472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 459/915] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:21.576436 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e9d1b-20d4-4a64-87ce-a0602bd6e880 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:21.576997 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e9d1b-20d4-4a64-87ce-a0602bd6e880 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:21.576997 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e9d1b-20d4-4a64-87ce-a0602bd6e880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:21.594854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-cf9e9d1b-20d4-4a64-87ce-a0602bd6e880 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:21.596070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cf9e9d1b-20d4-4a64-87ce-a0602bd6e880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:21.604388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c98b39-3f91-49eb-b259-0475f5c4cf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:21.604997 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c98b39-3f91-49eb-b259-0475f5c4cf38 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:21.605456 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c98b39-3f91-49eb-b259-0475f5c4cf38 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:21.605733 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c98b39-3f91-49eb-b259-0475f5c4cf38 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:21.606467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c98b39-3f91-49eb-b259-0475f5c4cf38 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:21.607935 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c98b39-3f91-49eb-b259-0475f5c4cf38 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:21.664513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 460/916] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:21.665387 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e9d1b-20d4-4a64-87ce-a0602bd6e880 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:21.687142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 457/917] 10.222.0.67 () {60 vars in 1141 bytes} [Fri Jun 24 15:01:21 2022] POST /identity/v3/services => generated 241 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:01:21.688781 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c98b39-3f91-49eb-b259-0475f5c4cf38 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:21.699697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:21.801111 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e9d1b-20d4-4a64-87ce-a0602bd6e880 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:21.830007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c98b39-3f91-49eb-b259-0475f5c4cf38 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:21.852761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095ed86-dbd7-4196-85e0-c76be63efd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:21.853838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095ed86-dbd7-4196-85e0-c76be63efd43 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:21.854190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095ed86-dbd7-4196-85e0-c76be63efd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:21.854594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095ed86-dbd7-4196-85e0-c76be63efd43 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:21.855366 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095ed86-dbd7-4196-85e0-c76be63efd43 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:21.855765 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095ed86-dbd7-4196-85e0-c76be63efd43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:21.857226 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095ed86-dbd7-4196-85e0-c76be63efd43 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:21.863481 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 461/918] 10.222.0.67 () {56 vars in 1126 bytes} [Fri Jun 24 15:01:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:21.869584 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095ed86-dbd7-4196-85e0-c76be63efd43 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:21.880741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d959bb-c2d1-4b70-83cc-433d672c4199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:21.881632 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d959bb-c2d1-4b70-83cc-433d672c4199 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:21.881919 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d959bb-c2d1-4b70-83cc-433d672c4199 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:21.882195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d959bb-c2d1-4b70-83cc-433d672c4199 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:21.899122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d959bb-c2d1-4b70-83cc-433d672c4199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f6d959bb-c2d1-4b70-83cc-433d672c4199 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:01:21.907791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:21.908734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:21.910183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:21.910183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:21.910183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:21.910183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:21.910183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:01:21.910183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:01:21.910183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:01:21.910183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:01:21.910183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:01:21.910183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:01:21.910183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 15:01:21.910183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:21.912790 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 458/919] 10.222.0.67 () {56 vars in 1120 bytes} [Fri Jun 24 15:01:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:21.920166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d959bb-c2d1-4b70-83cc-433d672c4199 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:21.937678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:21.989484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:21.996339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095ed86-dbd7-4196-85e0-c76be63efd43 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:22.036299 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490abf3-606b-454c-b0f2-c9eb1a7fcdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:22.037071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490abf3-606b-454c-b0f2-c9eb1a7fcdd6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:22.037429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490abf3-606b-454c-b0f2-c9eb1a7fcdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:22.037783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490abf3-606b-454c-b0f2-c9eb1a7fcdd6 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:22.038593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8490abf3-606b-454c-b0f2-c9eb1a7fcdd6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:22.039116 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8490abf3-606b-454c-b0f2-c9eb1a7fcdd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:22.040477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8490abf3-606b-454c-b0f2-c9eb1a7fcdd6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:22.046056 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d959bb-c2d1-4b70-83cc-433d672c4199 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:22.051002 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 462/920] 10.222.0.67 () {56 vars in 1126 bytes} [Fri Jun 24 15:01:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:22.052402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490abf3-606b-454c-b0f2-c9eb1a7fcdd6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:22.100222 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1050ff7-db79-44e1-8821-d7ceba1071ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:22.101167 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1050ff7-db79-44e1-8821-d7ceba1071ec None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:22.101671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1050ff7-db79-44e1-8821-d7ceba1071ec None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:22.102721 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1050ff7-db79-44e1-8821-d7ceba1071ec None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:22.103727 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1050ff7-db79-44e1-8821-d7ceba1071ec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:22.104727 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1050ff7-db79-44e1-8821-d7ceba1071ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:22.107027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1050ff7-db79-44e1-8821-d7ceba1071ec None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:22.119256 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 459/921] 10.222.0.67 () {56 vars in 1115 bytes} [Fri Jun 24 15:01:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:22.124884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1050ff7-db79-44e1-8821-d7ceba1071ec None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:22.187560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490abf3-606b-454c-b0f2-c9eb1a7fcdd6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:22.211662 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:22.225737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c986e-d6ab-4e64-91fd-f8d27b872e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:22.226403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c986e-d6ab-4e64-91fd-f8d27b872e31 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:22.226692 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c986e-d6ab-4e64-91fd-f8d27b872e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:22.226969 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c986e-d6ab-4e64-91fd-f8d27b872e31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:22.228302 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c986e-d6ab-4e64-91fd-f8d27b872e31 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:22.229118 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c986e-d6ab-4e64-91fd-f8d27b872e31 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:22.236799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1050ff7-db79-44e1-8821-d7ceba1071ec None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:22.271373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d501f-f0c3-4a61-835e-36d9fbfac464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:22.272168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d501f-f0c3-4a61-835e-36d9fbfac464 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:22.272447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d501f-f0c3-4a61-835e-36d9fbfac464 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:22.272718 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d501f-f0c3-4a61-835e-36d9fbfac464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:22.283076 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d501f-f0c3-4a61-835e-36d9fbfac464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:22.285187 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d501f-f0c3-4a61-835e-36d9fbfac464 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:22.295285 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 463/922] 10.222.0.67 () {60 vars in 1144 bytes} [Fri Jun 24 15:01:22 2022] POST /identity/v3/endpoints => generated 347 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:01:22.299321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165e0b0-a73b-4414-8084-7e5f4935b5e9 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:22.299321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165e0b0-a73b-4414-8084-7e5f4935b5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:22.299321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165e0b0-a73b-4414-8084-7e5f4935b5e9 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:22.299918 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 464/923] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:22.305431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1834fe-e945-4ad7-8341-1681147f5015 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:22.305723 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1834fe-e945-4ad7-8341-1681147f5015 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:22.306003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1834fe-e945-4ad7-8341-1681147f5015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:22.307904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-6e6d501f-f0c3-4a61-835e-36d9fbfac464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:22.307904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:22.307904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:22.307904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:22.307904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:22.307904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:22.307904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:22.307904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:22.307904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:22.307904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:22.307904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:22.307904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:22.307904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:22.307904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:22.307904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:22.308994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:22.310138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:22.310138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:22.310138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:22.310138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:22.317117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 460/924] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:22.318759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d501f-f0c3-4a61-835e-36d9fbfac464 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:22.326439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5e1834fe-e945-4ad7-8341-1681147f5015 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:22.327791 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5e1834fe-e945-4ad7-8341-1681147f5015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:22.338803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:22.427599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 465/925] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:22.448306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1834fe-e945-4ad7-8341-1681147f5015 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:22.454546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d501f-f0c3-4a61-835e-36d9fbfac464 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:22.487502 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:22.499316 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42863fe2-4d06-4447-9e70-2c4b488fb605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:22.502682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42863fe2-4d06-4447-9e70-2c4b488fb605 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:22.502682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42863fe2-4d06-4447-9e70-2c4b488fb605 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:22.502682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42863fe2-4d06-4447-9e70-2c4b488fb605 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:22.502682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42863fe2-4d06-4447-9e70-2c4b488fb605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:22.502682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42863fe2-4d06-4447-9e70-2c4b488fb605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:22.503819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42863fe2-4d06-4447-9e70-2c4b488fb605 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:22.520851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 461/926] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:22.526567 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42863fe2-4d06-4447-9e70-2c4b488fb605 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:22.575416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1834fe-e945-4ad7-8341-1681147f5015 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:22.612135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da6cc2-bc86-4b5e-8a28-00e29bb2ea11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:22.613643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da6cc2-bc86-4b5e-8a28-00e29bb2ea11 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:22.613928 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da6cc2-bc86-4b5e-8a28-00e29bb2ea11 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:22.614255 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da6cc2-bc86-4b5e-8a28-00e29bb2ea11 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:22.614923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da6cc2-bc86-4b5e-8a28-00e29bb2ea11 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:22.616356 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da6cc2-bc86-4b5e-8a28-00e29bb2ea11 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-25da6cc2-bc86-4b5e-8a28-00e29bb2ea11 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:22.628881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:22.629910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:22.630759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:22.630759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:22.630759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:22.630759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:22.630759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:22.630759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:22.630759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:22.630759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:22.630759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:22.630759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:22.630759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:22.630759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:22.630759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 15:01:22.630759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:22.630759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 466/927] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:01:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:22.635986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da6cc2-bc86-4b5e-8a28-00e29bb2ea11 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:22.653782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42863fe2-4d06-4447-9e70-2c4b488fb605 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:22.679919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:22.698614 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6ca58-912a-4470-8494-211f937b0294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:22.699808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6ca58-912a-4470-8494-211f937b0294 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:22.700357 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6ca58-912a-4470-8494-211f937b0294 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:22.700839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6ca58-912a-4470-8494-211f937b0294 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:22.709978 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6ca58-912a-4470-8494-211f937b0294 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:22.711750 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6ca58-912a-4470-8494-211f937b0294 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-6dc6ca58-912a-4470-8494-211f937b0294 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:01:22.729976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:22.730986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:22.731984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:22.731984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:22.731984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:01:22.731984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:01:22.731984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:01:22.731984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:01:22.731984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:01:22.731984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:01:22.731984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 15:01:22.731984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:22.740412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 462/928] 10.222.0.67 () {56 vars in 1117 bytes} [Fri Jun 24 15:01:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:22.748480 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6ca58-912a-4470-8494-211f937b0294 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:22.758162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da6cc2-bc86-4b5e-8a28-00e29bb2ea11 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:22.793562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb336a-49e9-45ce-8e8b-661b3ae97f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:22.794217 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb336a-49e9-45ce-8e8b-661b3ae97f4c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:22.794504 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb336a-49e9-45ce-8e8b-661b3ae97f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:22.794785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb336a-49e9-45ce-8e8b-661b3ae97f4c None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:22.795548 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb336a-49e9-45ce-8e8b-661b3ae97f4c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:22.795859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb336a-49e9-45ce-8e8b-661b3ae97f4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:22.797297 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb336a-49e9-45ce-8e8b-661b3ae97f4c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:22.801880 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:22.807632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 467/929] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:01:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:22.808845 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb336a-49e9-45ce-8e8b-661b3ae97f4c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:22.872557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6ca58-912a-4470-8494-211f937b0294 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:22.887609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:22.910971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27625691-2ac7-493d-a8ff-1aa940ba31a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:22.911693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27625691-2ac7-493d-a8ff-1aa940ba31a0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:22.911973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27625691-2ac7-493d-a8ff-1aa940ba31a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:22.912246 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27625691-2ac7-493d-a8ff-1aa940ba31a0 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:22.913084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27625691-2ac7-493d-a8ff-1aa940ba31a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:22.913390 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27625691-2ac7-493d-a8ff-1aa940ba31a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:22.914808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27625691-2ac7-493d-a8ff-1aa940ba31a0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:22.922903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb336a-49e9-45ce-8e8b-661b3ae97f4c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:22.935068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 463/930] 10.222.0.67 () {56 vars in 1149 bytes} [Fri Jun 24 15:01:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:22.940418 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27625691-2ac7-493d-a8ff-1aa940ba31a0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:22.961055 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a2e89-1ff0-4a5e-9257-6c4b8777889d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:22.961731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a2e89-1ff0-4a5e-9257-6c4b8777889d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:22.962017 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a2e89-1ff0-4a5e-9257-6c4b8777889d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:22.962296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a2e89-1ff0-4a5e-9257-6c4b8777889d None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:22.963016 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a2e89-1ff0-4a5e-9257-6c4b8777889d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:22.963359 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a2e89-1ff0-4a5e-9257-6c4b8777889d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:22.964734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a2e89-1ff0-4a5e-9257-6c4b8777889d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:22.970053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 468/931] 10.222.0.67 () {56 vars in 1114 bytes} [Fri Jun 24 15:01:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:22.975810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a2e89-1ff0-4a5e-9257-6c4b8777889d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:23.064333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27625691-2ac7-493d-a8ff-1aa940ba31a0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:23.074506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:23.090573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a2e89-1ff0-4a5e-9257-6c4b8777889d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:23.108878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c36a8-d86b-4f62-afb5-5367d8843376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:23.109926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c36a8-d86b-4f62-afb5-5367d8843376 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:23.110378 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c36a8-d86b-4f62-afb5-5367d8843376 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:23.110978 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c36a8-d86b-4f62-afb5-5367d8843376 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:23.124518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c36a8-d86b-4f62-afb5-5367d8843376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:23.126306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c36a8-d86b-4f62-afb5-5367d8843376 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:23.127931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:23.133504 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02661a5-e28f-4f94-b6e0-f51380eb4ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:23.134270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02661a5-e28f-4f94-b6e0-f51380eb4ddc None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:23.134625 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02661a5-e28f-4f94-b6e0-f51380eb4ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:23.134971 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02661a5-e28f-4f94-b6e0-f51380eb4ddc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:23.136320 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02661a5-e28f-4f94-b6e0-f51380eb4ddc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:23.137741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02661a5-e28f-4f94-b6e0-f51380eb4ddc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:23.149299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d39c36a8-d86b-4f62-afb5-5367d8843376 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:23.149299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:23.149299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:23.149299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:23.149299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:23.149299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:23.149299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:23.149299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:23.149299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:23.149299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:23.149299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:23.149299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:23.149299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:23.149299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:23.149299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:23.150412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:23.151381 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:23.151381 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:23.151381 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:23.151381 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:23.157001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 464/932] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:23.158053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c36a8-d86b-4f62-afb5-5367d8843376 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:23.267328 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c36a8-d86b-4f62-afb5-5367d8843376 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:23.302034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab66131-504c-4ee7-b0b5-9d3288e5ff1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:23.302683 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab66131-504c-4ee7-b0b5-9d3288e5ff1f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:23.302966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab66131-504c-4ee7-b0b5-9d3288e5ff1f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:23.303276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab66131-504c-4ee7-b0b5-9d3288e5ff1f None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:23.303992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab66131-504c-4ee7-b0b5-9d3288e5ff1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:23.304298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab66131-504c-4ee7-b0b5-9d3288e5ff1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:23.305644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab66131-504c-4ee7-b0b5-9d3288e5ff1f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:23.318557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 465/933] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:23.324211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab66131-504c-4ee7-b0b5-9d3288e5ff1f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:23.335608 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 469/934] 10.222.0.67 () {60 vars in 1144 bytes} [Fri Jun 24 15:01:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:01:23.345782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:23.442282 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab66131-504c-4ee7-b0b5-9d3288e5ff1f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:23.488957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1ead8-99a9-44df-8c95-fa835bb4df34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:23.489765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1ead8-99a9-44df-8c95-fa835bb4df34 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:23.490042 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1ead8-99a9-44df-8c95-fa835bb4df34 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:23.490307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1ead8-99a9-44df-8c95-fa835bb4df34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:23.497554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1ead8-99a9-44df-8c95-fa835bb4df34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:23.499000 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1ead8-99a9-44df-8c95-fa835bb4df34 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:23.514192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-eff1ead8-99a9-44df-8c95-fa835bb4df34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:23.514192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:23.514192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:23.514192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:23.514192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:23.514192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:23.514192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:23.514192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:23.514192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:23.514192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:23.514192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:23.514192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:23.514192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:01:23.514192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:01:23.514192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:23.515243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:23.516221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:01:23.516221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:01:23.516221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:01:23.516221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:01:23.516221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:01:23.516221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:01:23.516221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:23.516221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:23.516221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 466/935] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:23.523498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02661a5-e28f-4f94-b6e0-f51380eb4ddc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:23.638885 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02661a5-e28f-4f94-b6e0-f51380eb4ddc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:23.672790 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a234e-f068-484c-8386-36cda52c2c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:23.673451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a234e-f068-484c-8386-36cda52c2c0c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:23.673735 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a234e-f068-484c-8386-36cda52c2c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:23.674012 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a234e-f068-484c-8386-36cda52c2c0c None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:23.675352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a234e-f068-484c-8386-36cda52c2c0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:23.675662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a234e-f068-484c-8386-36cda52c2c0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:23.677353 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a234e-f068-484c-8386-36cda52c2c0c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:23.691354 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 470/936] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:01:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:23.697390 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1ead8-99a9-44df-8c95-fa835bb4df34 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:23.775796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:23.801151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1ead8-99a9-44df-8c95-fa835bb4df34 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:23.837138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491ea7a-e9d8-4ff0-8935-d1d040107de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:23.838035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491ea7a-e9d8-4ff0-8935-d1d040107de5 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:23.838316 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491ea7a-e9d8-4ff0-8935-d1d040107de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:23.838586 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491ea7a-e9d8-4ff0-8935-d1d040107de5 None admin] PATH_INFO: `/v3/projects/06da054e8b7545dfa56780ab56585be8/users/72f00ac4c6a247e485ac52a8dee4f1cb/roles/f70de858079b47cf92a3995fda64bbc0` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:23.840825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491ea7a-e9d8-4ff0-8935-d1d040107de5 None admin] RBAC: Authorizing `identity:create_grant(project_id=06da054e8b7545dfa56780ab56585be8, user_id=72f00ac4c6a247e485ac52a8dee4f1cb, role_id=f70de858079b47cf92a3995fda64bbc0)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:23.842245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491ea7a-e9d8-4ff0-8935-d1d040107de5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:23.873522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9586e-6e74-4ee2-9053-794e9fa5a226 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:23.873808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9586e-6e74-4ee2-9053-794e9fa5a226 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:23.874095 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9586e-6e74-4ee2-9053-794e9fa5a226 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:23.875626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 471/937] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:23.880389 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded870c-8b19-49ce-b9d9-06ba09c32b09 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:23.880673 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded870c-8b19-49ce-b9d9-06ba09c32b09 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:23.880947 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded870c-8b19-49ce-b9d9-06ba09c32b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:23.898605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9ded870c-8b19-49ce-b9d9-06ba09c32b09 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:23.900146 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9ded870c-8b19-49ce-b9d9-06ba09c32b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:23.909852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 467/938] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:01:23 2022] PUT /identity/v3/projects/06da054e8b7545dfa56780ab56585be8/users/72f00ac4c6a247e485ac52a8dee4f1cb/roles/f70de858079b47cf92a3995fda64bbc0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:01:23.967594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:23.989999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 472/939] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:24.014293 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491ea7a-e9d8-4ff0-8935-d1d040107de5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:24.130330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491ea7a-e9d8-4ff0-8935-d1d040107de5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:24.167643 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd9139-6ceb-40fd-bf10-22bec861fb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:24.168456 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd9139-6ceb-40fd-bf10-22bec861fb31 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:24.168740 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd9139-6ceb-40fd-bf10-22bec861fb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:24.169014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd9139-6ceb-40fd-bf10-22bec861fb31 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:24.169662 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd9139-6ceb-40fd-bf10-22bec861fb31 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:24.171027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd9139-6ceb-40fd-bf10-22bec861fb31 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:24.172865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-94fd9139-6ceb-40fd-bf10-22bec861fb31 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:24.184660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:24.185770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:24.187024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:24.187024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:24.187024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:24.187024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:24.187024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:24.187024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:24.187024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:24.187024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:24.187024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:24.187024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:24.187024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:24.187024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:24.187024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 15:01:24.187024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:24.187024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 468/940] 10.222.0.67 () {56 vars in 1126 bytes} [Fri Jun 24 15:01:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:24.196867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded870c-8b19-49ce-b9d9-06ba09c32b09 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:24.306185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded870c-8b19-49ce-b9d9-06ba09c32b09 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:24.341154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf82e9-9c5a-4823-9b0d-3da3f61f3d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:24.341907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf82e9-9c5a-4823-9b0d-3da3f61f3d49 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:24.342085 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf82e9-9c5a-4823-9b0d-3da3f61f3d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:24.342413 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf82e9-9c5a-4823-9b0d-3da3f61f3d49 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:24.343126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf82e9-9c5a-4823-9b0d-3da3f61f3d49 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:24.343754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf82e9-9c5a-4823-9b0d-3da3f61f3d49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:24.345224 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf82e9-9c5a-4823-9b0d-3da3f61f3d49 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:24.351244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 473/941] 10.222.0.67 () {56 vars in 1122 bytes} [Fri Jun 24 15:01:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:24.356855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd9139-6ceb-40fd-bf10-22bec861fb31 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:24.467891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd9139-6ceb-40fd-bf10-22bec861fb31 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:24.508821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecc7a8-c20e-4942-af51-070df0164782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:24.509493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecc7a8-c20e-4942-af51-070df0164782 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:24.509778 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecc7a8-c20e-4942-af51-070df0164782 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:24.510057 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecc7a8-c20e-4942-af51-070df0164782 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:24.510766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecc7a8-c20e-4942-af51-070df0164782 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:24.511072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecc7a8-c20e-4942-af51-070df0164782 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:24.512518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecc7a8-c20e-4942-af51-070df0164782 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:24.518208 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 469/942] 10.222.0.67 () {56 vars in 1122 bytes} [Fri Jun 24 15:01:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:24.522939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f5873-308f-4187-804e-72695c8ae01e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:24.523503 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f5873-308f-4187-804e-72695c8ae01e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:24.523784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f5873-308f-4187-804e-72695c8ae01e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:24.524250 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecc7a8-c20e-4942-af51-070df0164782 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:24.525616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 474/943] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:24.530554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a0e05-1d0b-48d7-a40c-c8642482b0eb None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:24.530839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a0e05-1d0b-48d7-a40c-c8642482b0eb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:24.531120 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a0e05-1d0b-48d7-a40c-c8642482b0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:24.549322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b50a0e05-1d0b-48d7-a40c-c8642482b0eb None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:24.551321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b50a0e05-1d0b-48d7-a40c-c8642482b0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:24.589653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:24.606535 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:24.642101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 475/944] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:24.652708 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecc7a8-c20e-4942-af51-070df0164782 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:24.663139 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a0e05-1d0b-48d7-a40c-c8642482b0eb None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:24.693661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea81b60-c449-462f-b60d-45199a1b5265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:24.694666 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea81b60-c449-462f-b60d-45199a1b5265 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:24.694959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea81b60-c449-462f-b60d-45199a1b5265 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:24.695275 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea81b60-c449-462f-b60d-45199a1b5265 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:24.696367 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea81b60-c449-462f-b60d-45199a1b5265 None admin] RBAC: Adding query filter params (interface=public, service_id=a08ab7c866184d109df58cb80e31e509, region_id=RegionOne) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:24.696683 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea81b60-c449-462f-b60d-45199a1b5265 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:24.698220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea81b60-c449-462f-b60d-45199a1b5265 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:24.709572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 470/945] 10.222.0.67 () {56 vars in 1251 bytes} [Fri Jun 24 15:01:24 2022] GET /identity/v3/endpoints?service_id=a08ab7c866184d109df58cb80e31e509&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:24.714867 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea81b60-c449-462f-b60d-45199a1b5265 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:24.775897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a0e05-1d0b-48d7-a40c-c8642482b0eb None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:24.814094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238abfe0-85f1-42d8-ab3e-979f46b1c1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:24.815389 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238abfe0-85f1-42d8-ab3e-979f46b1c1cf None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:24.816133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238abfe0-85f1-42d8-ab3e-979f46b1c1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:24.816611 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238abfe0-85f1-42d8-ab3e-979f46b1c1cf None admin] PATH_INFO: `/v3/services/compute` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:24.817506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238abfe0-85f1-42d8-ab3e-979f46b1c1cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:24.819049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea81b60-c449-462f-b60d-45199a1b5265 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:24.819283 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238abfe0-85f1-42d8-ab3e-979f46b1c1cf None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-238abfe0-85f1-42d8-ab3e-979f46b1c1cf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:24.832272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:24.833294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:24.834125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:24.834125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:24.834125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:24.834125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:24.834125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:24.834125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:24.834125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:24.834125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:24.834125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:24.834125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:24.834125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:24.834125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:24.834125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 15:01:24.834125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:24.841475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 476/946] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:24.842494 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238abfe0-85f1-42d8-ab3e-979f46b1c1cf None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:24.845524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:24.856091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78b5c4-ce87-4c41-bdb7-9f78eeb5f6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:24.860183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78b5c4-ce87-4c41-bdb7-9f78eeb5f6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:24.860183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78b5c4-ce87-4c41-bdb7-9f78eeb5f6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:24.860183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78b5c4-ce87-4c41-bdb7-9f78eeb5f6d8 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:24.860183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78b5c4-ce87-4c41-bdb7-9f78eeb5f6d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:24.860183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78b5c4-ce87-4c41-bdb7-9f78eeb5f6d8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:24.867591 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 471/947] 10.222.0.67 () {56 vars in 1087 bytes} [Fri Jun 24 15:01:24 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:01:24.883501 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:24.958095 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238abfe0-85f1-42d8-ab3e-979f46b1c1cf None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:24.996104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a08f6-dbf6-4f22-9934-ffcc8ad9a5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:24.997133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a08f6-dbf6-4f22-9934-ffcc8ad9a5fc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:24.997594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a08f6-dbf6-4f22-9934-ffcc8ad9a5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:24.998053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a08f6-dbf6-4f22-9934-ffcc8ad9a5fc None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:24.998977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a08f6-dbf6-4f22-9934-ffcc8ad9a5fc None admin] RBAC: Adding query filter params (name=compute) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:24.999494 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a08f6-dbf6-4f22-9934-ffcc8ad9a5fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:25.001072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a08f6-dbf6-4f22-9934-ffcc8ad9a5fc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:25.006809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 477/948] 10.222.0.67 () {56 vars in 1112 bytes} [Fri Jun 24 15:01:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:25.012913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78b5c4-ce87-4c41-bdb7-9f78eeb5f6d8 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:25.122782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78b5c4-ce87-4c41-bdb7-9f78eeb5f6d8 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:25.162327 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895b154-6787-48e1-9f96-2d80f0351812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:25.163320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895b154-6787-48e1-9f96-2d80f0351812 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:25.163797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895b154-6787-48e1-9f96-2d80f0351812 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:25.164261 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895b154-6787-48e1-9f96-2d80f0351812 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:25.165142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895b154-6787-48e1-9f96-2d80f0351812 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:25.165619 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895b154-6787-48e1-9f96-2d80f0351812 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:25.167165 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895b154-6787-48e1-9f96-2d80f0351812 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:25.172673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 472/949] 10.222.0.67 () {56 vars in 1112 bytes} [Fri Jun 24 15:01:25 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:25.502034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1e1b4-bf39-4283-a282-2d7d09fbdfa6 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:25.502554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1e1b4-bf39-4283-a282-2d7d09fbdfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:25.502699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1e1b4-bf39-4283-a282-2d7d09fbdfa6 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:25.504274 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 478/950] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:25.509727 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f2bb7-e73a-4af9-9b5e-19a6c760c604 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:25.510160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f2bb7-e73a-4af9-9b5e-19a6c760c604 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:25.510324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f2bb7-e73a-4af9-9b5e-19a6c760c604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:25.527266 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-af5f2bb7-e73a-4af9-9b5e-19a6c760c604 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:25.528452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-af5f2bb7-e73a-4af9-9b5e-19a6c760c604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:25.579381 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:25.599170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 473/951] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:25.623331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1e1b4-bf39-4283-a282-2d7d09fbdfa6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:25.727136 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:25.731798 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1e1b4-bf39-4283-a282-2d7d09fbdfa6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:25.767709 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ac579-3c6a-4488-a6ca-2462441e8cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:25.768725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ac579-3c6a-4488-a6ca-2462441e8cb6 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:25.769244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ac579-3c6a-4488-a6ca-2462441e8cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:25.769917 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ac579-3c6a-4488-a6ca-2462441e8cb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:25.770642 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ac579-3c6a-4488-a6ca-2462441e8cb6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:25.772168 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ac579-3c6a-4488-a6ca-2462441e8cb6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:25.773535 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: WARNING keystone.server.flask.common [None req-d03ac579-3c6a-4488-a6ca-2462441e8cb6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 24 15:01:25.814156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.sql.core [None req-d03ac579-3c6a-4488-a6ca-2462441e8cb6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 15:01:25.814156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 15:01:25.814156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [parameters: {'id': '9626cdde96164792a852403fb85d21ad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 15:01:25.814156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80523) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 15:01:25.823138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d03ac579-3c6a-4488-a6ca-2462441e8cb6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9626cdde96164792a852403fb85d21ad.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9626cdde96164792a852403fb85d21ad. Jun 24 15:01:25.823138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:25.823138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 15:01:25.823138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 15:01:25.823138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 15:01:25.823138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 15:01:25.823138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 15:01:25.823138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 15:01:25.823138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 15:01:25.823138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application conn.query(q) Jun 24 15:01:25.823138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 15:01:25.823138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 15:01:25.823138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result.read() Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 24 15:01:25.823981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application next(self.gen) Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application yield resource Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application next(self.gen) Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise self.value Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application session.commit() Jun 24 15:01:25.825241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.session.flush() Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application compat.raise_( Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 15:01:25.826154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 15:01:25.827081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 15:01:25.827081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 15:01:25.827081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 15:01:25.827081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 15:01:25.827081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 15:01:25.827081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 15:01:25.827081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 15:01:25.827081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 15:01:25.827081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 15:01:25.827081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 15:01:25.827081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 15:01:25.827081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 15:01:25.827081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 15:01:25.827081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 15:01:25.827081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application conn.query(q) Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application result.read() Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 15:01:25.829541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 15:01:25.831786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 15:01:25.831786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 15:01:25.831786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 15:01:25.831786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 15:01:25.831786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 15:01:25.831786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 15:01:25.831786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 15:01:25.831786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 15:01:25.831786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [parameters: {'id': '9626cdde96164792a852403fb85d21ad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 15:01:25.831786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 15:01:25.831786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:25.831786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:25.831786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:25.831786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:25.831786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:25.833782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:25.836086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:25.836086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 24 15:01:25.836086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 24 15:01:25.836086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:25.836086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:25.836086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 24 15:01:25.836086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 24 15:01:25.836086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9626cdde96164792a852403fb85d21ad. Jun 24 15:01:25.836086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:25.836086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 479/952] 10.222.0.67 () {60 vars in 1140 bytes} [Fri Jun 24 15:01:25 2022] POST /identity/v3/projects => generated 262 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:25.837399 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f2bb7-e73a-4af9-9b5e-19a6c760c604 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:25.936818 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f2bb7-e73a-4af9-9b5e-19a6c760c604 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:25.974266 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50d48f-3bdb-43e6-8d08-129bba2ec8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:25.975070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50d48f-3bdb-43e6-8d08-129bba2ec8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:25.975381 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50d48f-3bdb-43e6-8d08-129bba2ec8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:25.975656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50d48f-3bdb-43e6-8d08-129bba2ec8a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:25.982709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50d48f-3bdb-43e6-8d08-129bba2ec8a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:25.984169 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50d48f-3bdb-43e6-8d08-129bba2ec8a9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:25.998430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2a50d48f-3bdb-43e6-8d08-129bba2ec8a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:25.998430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:25.998430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:25.998430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:25.998430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:25.998430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:25.998430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:25.998430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:25.998430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:25.998430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:25.998430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:25.998430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:25.998430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:01:25.998430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:01:25.998430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:26.000607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:26.002799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:01:26.002799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:01:26.002799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:01:26.002799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:01:26.002799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:01:26.002799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:01:26.002799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:26.002799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:26.002799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 474/953] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:26.007200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ac579-3c6a-4488-a6ca-2462441e8cb6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:26.118348 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ac579-3c6a-4488-a6ca-2462441e8cb6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:26.154300 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd851943-e939-444d-8feb-0a96e9bf0200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:26.154967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd851943-e939-444d-8feb-0a96e9bf0200 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:26.155638 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd851943-e939-444d-8feb-0a96e9bf0200 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:26.155638 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd851943-e939-444d-8feb-0a96e9bf0200 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:26.156309 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd851943-e939-444d-8feb-0a96e9bf0200 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:26.156616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd851943-e939-444d-8feb-0a96e9bf0200 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:26.158242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd851943-e939-444d-8feb-0a96e9bf0200 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:26.162313 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e988dc0-f9d3-4b8b-9f1d-2453a711ed4f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:26.162599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e988dc0-f9d3-4b8b-9f1d-2453a711ed4f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:26.162875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e988dc0-f9d3-4b8b-9f1d-2453a711ed4f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:26.164516 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 475/954] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:01:26.171671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ba485-36bf-47c4-8b60-0e941cd991f9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:26.172574 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ba485-36bf-47c4-8b60-0e941cd991f9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:26.172968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ba485-36bf-47c4-8b60-0e941cd991f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:26.184098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 480/955] 10.222.0.67 () {56 vars in 1112 bytes} [Fri Jun 24 15:01:26 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:26.195943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-568ba485-36bf-47c4-8b60-0e941cd991f9 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:26.197524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-568ba485-36bf-47c4-8b60-0e941cd991f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:26.244968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:26.276992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 476/956] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:26.304391 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd851943-e939-444d-8feb-0a96e9bf0200 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:26.325489 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:26.418384 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd851943-e939-444d-8feb-0a96e9bf0200 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:26.455380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99983f-08e8-42b5-b02d-ea849caf8311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:26.456493 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99983f-08e8-42b5-b02d-ea849caf8311 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:26.456999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99983f-08e8-42b5-b02d-ea849caf8311 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:26.457416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99983f-08e8-42b5-b02d-ea849caf8311 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:26.468037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b99983f-08e8-42b5-b02d-ea849caf8311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-1b99983f-08e8-42b5-b02d-ea849caf8311 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:01:26.476992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:26.478180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:26.479267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:26.479267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:26.479267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:26.479267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:26.479267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:01:26.479267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:01:26.479267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:01:26.479267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:01:26.479267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:01:26.479267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:01:26.479267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 15:01:26.479267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:26.481681 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 481/957] 10.222.0.67 () {56 vars in 1120 bytes} [Fri Jun 24 15:01:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:26.490663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ba485-36bf-47c4-8b60-0e941cd991f9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:26.591788 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ba485-36bf-47c4-8b60-0e941cd991f9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:26.626601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0594b-efb8-42e6-adcc-e9c1920cb4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:26.627302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0594b-efb8-42e6-adcc-e9c1920cb4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:26.627616 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0594b-efb8-42e6-adcc-e9c1920cb4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:26.628421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0594b-efb8-42e6-adcc-e9c1920cb4b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:26.629227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0594b-efb8-42e6-adcc-e9c1920cb4b7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:26.629537 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0594b-efb8-42e6-adcc-e9c1920cb4b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:26.630905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0594b-efb8-42e6-adcc-e9c1920cb4b7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:26.639715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 477/958] 10.222.0.67 () {56 vars in 1115 bytes} [Fri Jun 24 15:01:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:26.645044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99983f-08e8-42b5-b02d-ea849caf8311 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:26.729649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:26.753875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99983f-08e8-42b5-b02d-ea849caf8311 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:26.791010 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9cc72-2f25-4148-ac0c-268447aaeef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:26.791845 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9cc72-2f25-4148-ac0c-268447aaeef2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:26.792137 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9cc72-2f25-4148-ac0c-268447aaeef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:26.792410 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9cc72-2f25-4148-ac0c-268447aaeef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:26.804185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9cc72-2f25-4148-ac0c-268447aaeef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:26.805640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9cc72-2f25-4148-ac0c-268447aaeef2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:26.823536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-45d9cc72-2f25-4148-ac0c-268447aaeef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:26.823536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:26.823536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:26.823536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:26.823536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:26.823536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:26.823536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:26.823536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:26.823536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:26.823536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:26.823536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:26.823536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:26.823536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:26.823536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:26.823536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:26.825739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:26.827562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:26.827562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:26.827562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:26.827562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:26.827562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 482/959] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:26.831240 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0594b-efb8-42e6-adcc-e9c1920cb4b7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:26.873377 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:26.942222 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0594b-efb8-42e6-adcc-e9c1920cb4b7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:26.978030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48d6e1-6641-4c48-aeba-7f7f5b2ec198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:26.978700 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48d6e1-6641-4c48-aeba-7f7f5b2ec198 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:26.978985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48d6e1-6641-4c48-aeba-7f7f5b2ec198 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:26.979288 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48d6e1-6641-4c48-aeba-7f7f5b2ec198 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:26.980073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef48d6e1-6641-4c48-aeba-7f7f5b2ec198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:26.980324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef48d6e1-6641-4c48-aeba-7f7f5b2ec198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:26.981684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef48d6e1-6641-4c48-aeba-7f7f5b2ec198 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:26.994668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 478/960] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:26.999873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9cc72-2f25-4148-ac0c-268447aaeef2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:27.032662 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859d0c6-8cf8-4431-a0ee-a54eb0a08d88 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:27.032954 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859d0c6-8cf8-4431-a0ee-a54eb0a08d88 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:27.033241 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859d0c6-8cf8-4431-a0ee-a54eb0a08d88 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:27.035722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 479/961] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:27.042706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfad17a-caf9-498a-b7e1-7db39482ea54 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:27.043054 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfad17a-caf9-498a-b7e1-7db39482ea54 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:27.043369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfad17a-caf9-498a-b7e1-7db39482ea54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:27.061369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-fcfad17a-caf9-498a-b7e1-7db39482ea54 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:27.062627 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fcfad17a-caf9-498a-b7e1-7db39482ea54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:27.143834 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9cc72-2f25-4148-ac0c-268447aaeef2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:27.150325 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 480/962] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:27.173726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:27.176862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfad17a-caf9-498a-b7e1-7db39482ea54 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:27.183561 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a1bcd-13bf-456a-9173-6454ce321dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:27.184421 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a1bcd-13bf-456a-9173-6454ce321dfd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:27.184714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a1bcd-13bf-456a-9173-6454ce321dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:27.184993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a1bcd-13bf-456a-9173-6454ce321dfd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:27.193536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a1bcd-13bf-456a-9173-6454ce321dfd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:27.195564 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a1bcd-13bf-456a-9173-6454ce321dfd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:27.208926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-800a1bcd-13bf-456a-9173-6454ce321dfd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:01:27.210710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:27.211932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:27.213338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:27.213338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:27.213338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:01:27.213338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:01:27.213338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:01:27.213338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:01:27.213338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:01:27.213338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:01:27.213338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 15:01:27.213338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:27.213338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 483/963] 10.222.0.67 () {56 vars in 1117 bytes} [Fri Jun 24 15:01:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:27.217513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a1bcd-13bf-456a-9173-6454ce321dfd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:27.312218 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfad17a-caf9-498a-b7e1-7db39482ea54 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:27.332161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a1bcd-13bf-456a-9173-6454ce321dfd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:27.350466 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726d1ad-ee3a-4ec7-9bd5-df452977cecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:27.351324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726d1ad-ee3a-4ec7-9bd5-df452977cecd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:27.351604 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726d1ad-ee3a-4ec7-9bd5-df452977cecd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:27.351893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726d1ad-ee3a-4ec7-9bd5-df452977cecd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:27.352553 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726d1ad-ee3a-4ec7-9bd5-df452977cecd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:27.353906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726d1ad-ee3a-4ec7-9bd5-df452977cecd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2726d1ad-ee3a-4ec7-9bd5-df452977cecd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:27.366424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:27.367886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:27.368759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:27.368759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:27.368759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:27.368759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:27.368759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:27.368759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:27.368759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:27.368759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:27.368759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:27.368759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:27.368759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:27.368759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:27.368759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 15:01:27.368759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:27.368759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 481/964] 10.222.0.67 () {56 vars in 1126 bytes} [Fri Jun 24 15:01:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:27.369585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b43ade-52bf-43d4-b31a-29266710a36f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:27.369585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b43ade-52bf-43d4-b31a-29266710a36f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:27.370213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b43ade-52bf-43d4-b31a-29266710a36f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:27.371193 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 482/965] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:27.373471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b43ade-52bf-43d4-b31a-29266710a36f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:27.375634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4f8b4-478b-447b-ad3d-49cb3eeb8e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:27.376395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4f8b4-478b-447b-ad3d-49cb3eeb8e41 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:27.376685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4f8b4-478b-447b-ad3d-49cb3eeb8e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:27.376963 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4f8b4-478b-447b-ad3d-49cb3eeb8e41 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:27.378418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4f8b4-478b-447b-ad3d-49cb3eeb8e41 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:27.378732 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4f8b4-478b-447b-ad3d-49cb3eeb8e41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:27.380179 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4f8b4-478b-447b-ad3d-49cb3eeb8e41 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:27.398422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 484/966] 10.222.0.67 () {56 vars in 1149 bytes} [Fri Jun 24 15:01:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:27.399861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124967b-b0a9-467e-880b-530f8dcf4e2c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:27.400155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124967b-b0a9-467e-880b-530f8dcf4e2c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:27.400437 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124967b-b0a9-467e-880b-530f8dcf4e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:27.417384 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6124967b-b0a9-467e-880b-530f8dcf4e2c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:27.418876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6124967b-b0a9-467e-880b-530f8dcf4e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:27.466983 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:27.482309 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:27.494894 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b43ade-52bf-43d4-b31a-29266710a36f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:27.499325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 485/967] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:27.500471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124967b-b0a9-467e-880b-530f8dcf4e2c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:27.537069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749020a-8505-40db-8241-84584fe352bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:27.538061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749020a-8505-40db-8241-84584fe352bb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:27.538346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749020a-8505-40db-8241-84584fe352bb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:27.538623 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749020a-8505-40db-8241-84584fe352bb None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:27.539412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749020a-8505-40db-8241-84584fe352bb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:27.539771 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749020a-8505-40db-8241-84584fe352bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:27.541146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749020a-8505-40db-8241-84584fe352bb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:27.546030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 483/968] 10.222.0.67 () {56 vars in 1122 bytes} [Fri Jun 24 15:01:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:27.546796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749020a-8505-40db-8241-84584fe352bb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:27.619333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124967b-b0a9-467e-880b-530f8dcf4e2c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:27.656822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8a0db-55c9-4a73-b2f6-071a9a61a991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:27.657570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8a0db-55c9-4a73-b2f6-071a9a61a991 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:27.657851 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8a0db-55c9-4a73-b2f6-071a9a61a991 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:27.658133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8a0db-55c9-4a73-b2f6-071a9a61a991 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:27.660017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749020a-8505-40db-8241-84584fe352bb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:27.697905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf29e5d-0551-4688-b366-f5f72c564233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:27.698570 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf29e5d-0551-4688-b366-f5f72c564233 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:27.698856 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf29e5d-0551-4688-b366-f5f72c564233 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:27.699135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf29e5d-0551-4688-b366-f5f72c564233 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:27.699879 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf29e5d-0551-4688-b366-f5f72c564233 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:27.701239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf29e5d-0551-4688-b366-f5f72c564233 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:27.705468 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:27.726736 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8a0db-55c9-4a73-b2f6-071a9a61a991 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:27.728204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8a0db-55c9-4a73-b2f6-071a9a61a991 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:27.799213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 484/969] 10.222.0.67 () {60 vars in 1141 bytes} [Fri Jun 24 15:01:27 2022] POST /identity/v3/services => generated 242 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:01:27.799213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf29e5d-0551-4688-b366-f5f72c564233 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:27.817553 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:27.844347 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 486/970] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:01:27 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:01:27.850989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8a0db-55c9-4a73-b2f6-071a9a61a991 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:27.948833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:27.952770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf29e5d-0551-4688-b366-f5f72c564233 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:27.976145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8a0db-55c9-4a73-b2f6-071a9a61a991 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:28.003692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d400a-eb85-424b-8c8a-dc06dcb51ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:28.004648 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d400a-eb85-424b-8c8a-dc06dcb51ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:28.004924 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d400a-eb85-424b-8c8a-dc06dcb51ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:28.005197 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d400a-eb85-424b-8c8a-dc06dcb51ff2 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:28.005900 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d400a-eb85-424b-8c8a-dc06dcb51ff2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:28.006201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d400a-eb85-424b-8c8a-dc06dcb51ff2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:28.008580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d400a-eb85-424b-8c8a-dc06dcb51ff2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:28.015294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 485/971] 10.222.0.67 () {56 vars in 1122 bytes} [Fri Jun 24 15:01:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:28.016473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db22c76-a679-4e54-8b8a-4e262bf2ad71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:28.017272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db22c76-a679-4e54-8b8a-4e262bf2ad71 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:28.017558 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db22c76-a679-4e54-8b8a-4e262bf2ad71 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:28.017834 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db22c76-a679-4e54-8b8a-4e262bf2ad71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:28.022150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d400a-eb85-424b-8c8a-dc06dcb51ff2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:28.030825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db22c76-a679-4e54-8b8a-4e262bf2ad71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:28.032339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db22c76-a679-4e54-8b8a-4e262bf2ad71 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:28.054998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9db22c76-a679-4e54-8b8a-4e262bf2ad71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:28.054998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:28.054998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:28.054998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:28.054998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:28.054998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:28.054998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:28.054998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:28.054998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:28.054998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:28.054998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:28.054998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:28.054998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:28.054998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:28.054998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:28.056018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:28.057109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:28.057109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:28.057109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:28.057109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:28.057109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 487/972] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:28.061908 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db22c76-a679-4e54-8b8a-4e262bf2ad71 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:28.123696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:28.165767 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d400a-eb85-424b-8c8a-dc06dcb51ff2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:28.189643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db22c76-a679-4e54-8b8a-4e262bf2ad71 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:28.214523 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f378e22-fe26-4bb2-9e18-bc1fc65f36f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:28.215721 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f378e22-fe26-4bb2-9e18-bc1fc65f36f0 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:28.216236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f378e22-fe26-4bb2-9e18-bc1fc65f36f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:28.217143 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f378e22-fe26-4bb2-9e18-bc1fc65f36f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:28.218327 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f378e22-fe26-4bb2-9e18-bc1fc65f36f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:28.220129 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:28.223154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f378e22-fe26-4bb2-9e18-bc1fc65f36f0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:28.232653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d3990-0628-4289-8410-a0d708da4354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:28.236440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d3990-0628-4289-8410-a0d708da4354 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:28.236440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d3990-0628-4289-8410-a0d708da4354 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:28.236440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d3990-0628-4289-8410-a0d708da4354 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:28.236440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d3990-0628-4289-8410-a0d708da4354 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:28.237218 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d3990-0628-4289-8410-a0d708da4354 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:28.239860 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d3990-0628-4289-8410-a0d708da4354 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:28.252184 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 488/973] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:28.257432 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d3990-0628-4289-8410-a0d708da4354 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:28.330589 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 486/974] 10.222.0.67 () {60 vars in 1144 bytes} [Fri Jun 24 15:01:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:01:28.372192 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d3990-0628-4289-8410-a0d708da4354 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:28.418248 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa6c6f-758b-4e18-8662-0b612827a028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:28.419054 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa6c6f-758b-4e18-8662-0b612827a028 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:28.419369 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa6c6f-758b-4e18-8662-0b612827a028 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:28.419695 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa6c6f-758b-4e18-8662-0b612827a028 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:28.422225 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203cb5f-3c5f-4692-9dd2-ad1f7860c113 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:28.422515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203cb5f-3c5f-4692-9dd2-ad1f7860c113 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:28.422795 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203cb5f-3c5f-4692-9dd2-ad1f7860c113 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:28.424555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 487/975] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:01:28.428302 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa6c6f-758b-4e18-8662-0b612827a028 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:28.430014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa6c6f-758b-4e18-8662-0b612827a028 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:28.430627 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed88551-50c6-4bb3-acdd-0bb07cf4825e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:28.430891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed88551-50c6-4bb3-acdd-0bb07cf4825e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:28.431166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed88551-50c6-4bb3-acdd-0bb07cf4825e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:28.447414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-51aa6c6f-758b-4e18-8662-0b612827a028 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:28.447414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:28.447414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:28.447414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:28.447414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:28.447414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:28.447414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:28.447414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:28.447414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:28.447414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:28.447414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:28.447414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:28.447414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:01:28.447414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:01:28.447414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:28.448505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:28.449407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:01:28.449407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:01:28.449407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:01:28.449407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:01:28.449407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:01:28.449407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:01:28.449407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:28.449407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:28.449848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7ed88551-50c6-4bb3-acdd-0bb07cf4825e None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:28.450652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7ed88551-50c6-4bb3-acdd-0bb07cf4825e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:28.451904 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 489/976] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:28.463212 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa6c6f-758b-4e18-8662-0b612827a028 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:28.512700 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:28.552177 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 488/977] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:28.591977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed88551-50c6-4bb3-acdd-0bb07cf4825e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:28.597736 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa6c6f-758b-4e18-8662-0b612827a028 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:28.648521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052236ec-4226-4e78-b7cd-921d06c21da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:28.649915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052236ec-4226-4e78-b7cd-921d06c21da2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:28.650401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052236ec-4226-4e78-b7cd-921d06c21da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:28.650876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052236ec-4226-4e78-b7cd-921d06c21da2 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:28.652499 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052236ec-4226-4e78-b7cd-921d06c21da2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:28.653072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052236ec-4226-4e78-b7cd-921d06c21da2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:28.655254 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052236ec-4226-4e78-b7cd-921d06c21da2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:28.660136 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:28.670981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 490/978] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:01:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:28.677980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052236ec-4226-4e78-b7cd-921d06c21da2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:28.719504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed88551-50c6-4bb3-acdd-0bb07cf4825e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:28.777001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28231e-bd4f-4c78-a7cf-f6be81879760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:28.777748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28231e-bd4f-4c78-a7cf-f6be81879760 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:28.778036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28231e-bd4f-4c78-a7cf-f6be81879760 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:28.778317 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28231e-bd4f-4c78-a7cf-f6be81879760 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:28.779019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28231e-bd4f-4c78-a7cf-f6be81879760 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:28.781268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28231e-bd4f-4c78-a7cf-f6be81879760 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:28.782909 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: WARNING keystone.server.flask.common [None req-ba28231e-bd4f-4c78-a7cf-f6be81879760 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 24 15:01:28.800379 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052236ec-4226-4e78-b7cd-921d06c21da2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:28.803994 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:28.834670 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f1464-ffb6-4702-b8ba-dd1297e6a5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:28.835353 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f1464-ffb6-4702-b8ba-dd1297e6a5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:28.835641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f1464-ffb6-4702-b8ba-dd1297e6a5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:28.835914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f1464-ffb6-4702-b8ba-dd1297e6a5b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:28.836737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f1464-ffb6-4702-b8ba-dd1297e6a5b4 None admin] RBAC: Adding query filter params (role.id=f70de858079b47cf92a3995fda64bbc0, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=72f00ac4c6a247e485ac52a8dee4f1cb) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:28.837047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f1464-ffb6-4702-b8ba-dd1297e6a5b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:28.837808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.sql.core [None req-ba28231e-bd4f-4c78-a7cf-f6be81879760 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 15:01:28.837808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 15:01:28.837808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [parameters: {'user_id': 'c070c45cf59f41ed8b2739ac5c5dca53', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 15:01:28.837808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80522) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 15:01:28.838466 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f1464-ffb6-4702-b8ba-dd1297e6a5b4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:28.851232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 491/979] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:01:28 2022] GET /identity/v3/role_assignments?user.id=72f00ac4c6a247e485ac52a8dee4f1cb&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=f70de858079b47cf92a3995fda64bbc0 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:28.869038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-ba28231e-bd4f-4c78-a7cf-f6be81879760 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 15:01:28.869038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:28.869038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 15:01:28.869038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 15:01:28.869038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 15:01:28.869038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 15:01:28.869038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 15:01:28.869038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 15:01:28.869038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 15:01:28.869038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application conn.query(q) Jun 24 15:01:28.869038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 15:01:28.869038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 15:01:28.869038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 15:01:28.869038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result.read() Jun 24 15:01:28.869038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 15:01:28.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application next(self.gen) Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application yield resource Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application next(self.gen) Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise self.value Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application session.commit() Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 15:01:28.871430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.session.flush() Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application compat.raise_( Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 15:01:28.872407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 15:01:28.873311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application conn.query(q) Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result.read() Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 15:01:28.875064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 15:01:28.877084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 15:01:28.877084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 15:01:28.877084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 15:01:28.877084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 15:01:28.877084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 15:01:28.877084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 15:01:28.877084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c070c45cf59f41ed8b2739ac5c5dca53', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 15:01:28.877084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 15:01:28.877084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:28.877084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:28.877084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:28.877084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:28.877084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:28.877084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:28.877084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 24 15:01:28.879898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 24 15:01:28.883836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 24 15:01:28.883836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 24 15:01:28.883836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 15:01:28.883836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 15:01:28.883836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 15:01:28.883836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:28.883836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 489/980] 10.222.0.67 () {60 vars in 1131 bytes} [Fri Jun 24 15:01:28 2022] POST /identity/v3/users => generated 160 bytes in 281 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:28.884742 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f1464-ffb6-4702-b8ba-dd1297e6a5b4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:29.001585 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f1464-ffb6-4702-b8ba-dd1297e6a5b4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:29.040552 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e21d66-a046-4134-b2a1-32f01096eb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:29.041697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e21d66-a046-4134-b2a1-32f01096eb82 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:29.042013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e21d66-a046-4134-b2a1-32f01096eb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:29.042278 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e21d66-a046-4134-b2a1-32f01096eb82 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:29.053408 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e21d66-a046-4134-b2a1-32f01096eb82 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:29.054222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e21d66-a046-4134-b2a1-32f01096eb82 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-71e21d66-a046-4134-b2a1-32f01096eb82 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:01:29.070689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:29.071761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:29.072743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:29.072743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:29.072743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:01:29.072743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:01:29.072743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:01:29.072743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:01:29.072743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:01:29.072743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:01:29.072743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 15:01:29.072743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:29.072743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 492/981] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:01:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:29.078106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28231e-bd4f-4c78-a7cf-f6be81879760 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:29.189601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28231e-bd4f-4c78-a7cf-f6be81879760 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:29.214098 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:29.225597 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadae76a-17b6-45cb-9ea2-6f5b1849497d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:29.226271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadae76a-17b6-45cb-9ea2-6f5b1849497d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:29.226552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadae76a-17b6-45cb-9ea2-6f5b1849497d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:29.226828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadae76a-17b6-45cb-9ea2-6f5b1849497d None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:29.227690 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadae76a-17b6-45cb-9ea2-6f5b1849497d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:29.227969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadae76a-17b6-45cb-9ea2-6f5b1849497d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:29.244511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadae76a-17b6-45cb-9ea2-6f5b1849497d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:29.269002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 490/982] 10.222.0.67 () {56 vars in 1101 bytes} [Fri Jun 24 15:01:29 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:30.004409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434a6df-018c-4128-8c6f-18b0a94bdb30 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:30.004934 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434a6df-018c-4128-8c6f-18b0a94bdb30 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:30.005045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434a6df-018c-4128-8c6f-18b0a94bdb30 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:30.006495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 493/983] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:30.011763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf04c9-0e66-490b-8e79-4dd30d92ea8d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:30.012163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf04c9-0e66-490b-8e79-4dd30d92ea8d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:30.012325 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf04c9-0e66-490b-8e79-4dd30d92ea8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:30.030109 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-30cf04c9-0e66-490b-8e79-4dd30d92ea8d None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:30.031660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-30cf04c9-0e66-490b-8e79-4dd30d92ea8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:30.111342 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 491/984] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:30.134903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434a6df-018c-4128-8c6f-18b0a94bdb30 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:30.169103 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:30.244074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434a6df-018c-4128-8c6f-18b0a94bdb30 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:30.277506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffbc4f-2c50-4d83-afec-a35b659c7836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:30.278228 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffbc4f-2c50-4d83-afec-a35b659c7836 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:30.278509 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffbc4f-2c50-4d83-afec-a35b659c7836 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:30.278781 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffbc4f-2c50-4d83-afec-a35b659c7836 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:30.279455 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dffbc4f-2c50-4d83-afec-a35b659c7836 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:30.280808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dffbc4f-2c50-4d83-afec-a35b659c7836 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-5dffbc4f-2c50-4d83-afec-a35b659c7836 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:30.293205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:30.294237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:30.295097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:30.295097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:30.295097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:30.295097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:30.295097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:30.295097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:30.295097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:30.295097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:30.295097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:30.295097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:30.295097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:30.295097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:30.295097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 15:01:30.295097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:30.295097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 494/985] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:30.300974 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf04c9-0e66-490b-8e79-4dd30d92ea8d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:30.400718 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf04c9-0e66-490b-8e79-4dd30d92ea8d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:30.411942 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:30.435772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b4c15-432e-4717-bdae-1c48e7fa72cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:30.436439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b4c15-432e-4717-bdae-1c48e7fa72cc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:30.436719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b4c15-432e-4717-bdae-1c48e7fa72cc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:30.436991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b4c15-432e-4717-bdae-1c48e7fa72cc None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:30.437686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b4c15-432e-4717-bdae-1c48e7fa72cc None admin] RBAC: Adding query filter params (name=compute) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:30.437988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b4c15-432e-4717-bdae-1c48e7fa72cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:30.439354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b4c15-432e-4717-bdae-1c48e7fa72cc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:30.449735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 492/986] 10.222.0.67 () {56 vars in 1112 bytes} [Fri Jun 24 15:01:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:30.451385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffbc4f-2c50-4d83-afec-a35b659c7836 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:30.532655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adf2c8-2d44-4bdc-83ed-dcba8988a2ec None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:30.532837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adf2c8-2d44-4bdc-83ed-dcba8988a2ec None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:30.533118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adf2c8-2d44-4bdc-83ed-dcba8988a2ec None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:30.536781 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 493/987] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:30.542550 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f627259-6261-4cb7-959d-da63b7fdc6a2 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:30.542844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f627259-6261-4cb7-959d-da63b7fdc6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:30.543131 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f627259-6261-4cb7-959d-da63b7fdc6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:30.547777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:30.558414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffbc4f-2c50-4d83-afec-a35b659c7836 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:30.561770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5f627259-6261-4cb7-959d-da63b7fdc6a2 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:30.562964 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5f627259-6261-4cb7-959d-da63b7fdc6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:30.593912 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4e43c-5a49-4309-a46f-2ec880d72852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:30.594670 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4e43c-5a49-4309-a46f-2ec880d72852 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:30.594958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4e43c-5a49-4309-a46f-2ec880d72852 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:30.595272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4e43c-5a49-4309-a46f-2ec880d72852 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:30.595983 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4e43c-5a49-4309-a46f-2ec880d72852 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:30.596850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4e43c-5a49-4309-a46f-2ec880d72852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:30.598227 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4e43c-5a49-4309-a46f-2ec880d72852 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:30.607904 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 495/988] 10.222.0.67 () {56 vars in 1112 bytes} [Fri Jun 24 15:01:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:30.609210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4e43c-5a49-4309-a46f-2ec880d72852 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:30.640186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 494/989] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:30.663934 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f627259-6261-4cb7-959d-da63b7fdc6a2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:30.720989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4e43c-5a49-4309-a46f-2ec880d72852 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:30.758042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c8267-9dd4-45b4-aa5f-ef2dcfe6f18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:30.758826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c8267-9dd4-45b4-aa5f-ef2dcfe6f18d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:30.759132 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c8267-9dd4-45b4-aa5f-ef2dcfe6f18d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:30.759520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c8267-9dd4-45b4-aa5f-ef2dcfe6f18d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:30.760300 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c8267-9dd4-45b4-aa5f-ef2dcfe6f18d None admin] RBAC: Adding query filter params (interface=public, service_id=0a84c2a24eec4b5998831283b2ffb1e6, region_id=RegionOne) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:30.760612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c8267-9dd4-45b4-aa5f-ef2dcfe6f18d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:30.762092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c8267-9dd4-45b4-aa5f-ef2dcfe6f18d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:30.765490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f627259-6261-4cb7-959d-da63b7fdc6a2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:30.767854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:30.773689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 496/990] 10.222.0.67 () {56 vars in 1251 bytes} [Fri Jun 24 15:01:30 2022] GET /identity/v3/endpoints?service_id=0a84c2a24eec4b5998831283b2ffb1e6&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:30.774444 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c8267-9dd4-45b4-aa5f-ef2dcfe6f18d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:30.798820 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:30.803075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db60782-38ef-4a94-ad1b-c8a089d72810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:30.803935 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db60782-38ef-4a94-ad1b-c8a089d72810 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:30.804231 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db60782-38ef-4a94-ad1b-c8a089d72810 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:30.804497 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db60782-38ef-4a94-ad1b-c8a089d72810 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:30.805146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db60782-38ef-4a94-ad1b-c8a089d72810 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:30.806502 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db60782-38ef-4a94-ad1b-c8a089d72810 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-7db60782-38ef-4a94-ad1b-c8a089d72810 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:30.818409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:30.819495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:30.820358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:30.820358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:30.820358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:30.820358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:30.820358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:30.820358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:30.820358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:30.820358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:30.820358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:30.820358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:30.820358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:30.820358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:30.820358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 15:01:30.820358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:30.823335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 495/991] 10.222.0.67 () {56 vars in 1126 bytes} [Fri Jun 24 15:01:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:30.825764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db60782-38ef-4a94-ad1b-c8a089d72810 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:30.886327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c8267-9dd4-45b4-aa5f-ef2dcfe6f18d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:30.922739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00d345-8f33-4960-85c6-e9e7d6bd2505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:30.923438 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00d345-8f33-4960-85c6-e9e7d6bd2505 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:30.923728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00d345-8f33-4960-85c6-e9e7d6bd2505 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:30.924006 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00d345-8f33-4960-85c6-e9e7d6bd2505 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:30.925304 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00d345-8f33-4960-85c6-e9e7d6bd2505 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:30.926995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00d345-8f33-4960-85c6-e9e7d6bd2505 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:30.931959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db60782-38ef-4a94-ad1b-c8a089d72810 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:30.933770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 497/992] 10.222.0.67 () {56 vars in 1087 bytes} [Fri Jun 24 15:01:30 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:01:30.970998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed177c-8e8a-4749-bf62-81c858e459a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:30.972333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed177c-8e8a-4749-bf62-81c858e459a2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:30.972620 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed177c-8e8a-4749-bf62-81c858e459a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:30.972896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed177c-8e8a-4749-bf62-81c858e459a2 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:30.973611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed177c-8e8a-4749-bf62-81c858e459a2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:30.973918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed177c-8e8a-4749-bf62-81c858e459a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:30.975335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed177c-8e8a-4749-bf62-81c858e459a2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:30.985714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 496/993] 10.222.0.67 () {56 vars in 1122 bytes} [Fri Jun 24 15:01:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:30.988043 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00d345-8f33-4960-85c6-e9e7d6bd2505 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:31.010486 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243db87-b8e4-42f5-81b6-ed5828e295b0 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:31.010815 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243db87-b8e4-42f5-81b6-ed5828e295b0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:31.011045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243db87-b8e4-42f5-81b6-ed5828e295b0 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:31.013276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 497/994] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:31.019619 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1eb3a5-2e2e-4aad-99f3-e2fcfd737858 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:31.020106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1eb3a5-2e2e-4aad-99f3-e2fcfd737858 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:31.020382 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1eb3a5-2e2e-4aad-99f3-e2fcfd737858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:31.040177 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1c1eb3a5-2e2e-4aad-99f3-e2fcfd737858 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:31.041421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1c1eb3a5-2e2e-4aad-99f3-e2fcfd737858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:31.072748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:31.089834 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:31.112110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00d345-8f33-4960-85c6-e9e7d6bd2505 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:31.125228 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 498/995] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:31 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:31.147689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1eb3a5-2e2e-4aad-99f3-e2fcfd737858 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:31.152327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e29c8-149d-46cc-833f-780a25ec5c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:31.153007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e29c8-149d-46cc-833f-780a25ec5c2b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:31.153294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e29c8-149d-46cc-833f-780a25ec5c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:31.153574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e29c8-149d-46cc-833f-780a25ec5c2b None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:31.154286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e29c8-149d-46cc-833f-780a25ec5c2b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:31.154599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e29c8-149d-46cc-833f-780a25ec5c2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:31.156014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e29c8-149d-46cc-833f-780a25ec5c2b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:31.161822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 498/996] 10.222.0.67 () {56 vars in 1122 bytes} [Fri Jun 24 15:01:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:31.167532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e29c8-149d-46cc-833f-780a25ec5c2b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:31.259291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1eb3a5-2e2e-4aad-99f3-e2fcfd737858 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:31.289891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e29c8-149d-46cc-833f-780a25ec5c2b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:31.294929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64739a3-ed03-411f-bd6f-27d1eb870980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:31.295832 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64739a3-ed03-411f-bd6f-27d1eb870980 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:31.296179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64739a3-ed03-411f-bd6f-27d1eb870980 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:31.296523 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64739a3-ed03-411f-bd6f-27d1eb870980 None admin] PATH_INFO: `/v3/services/image` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:31.297238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64739a3-ed03-411f-bd6f-27d1eb870980 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:31.298664 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64739a3-ed03-411f-bd6f-27d1eb870980 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a64739a3-ed03-411f-bd6f-27d1eb870980 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:31.310932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:31.312260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:31.313307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:31.313307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:31.313307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:31.313307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:31.313307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:31.313307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:31.313307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:31.313307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:31.313307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:31.313307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:31.313307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:31.313307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:31.313307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 15:01:31.313307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:31.314631 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 499/997] 10.222.0.67 () {56 vars in 1105 bytes} [Fri Jun 24 15:01:31 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:31.320169 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64739a3-ed03-411f-bd6f-27d1eb870980 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:31.329007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568c738-b3cc-4aa9-a34c-d594d3fb571e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:31.329679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568c738-b3cc-4aa9-a34c-d594d3fb571e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:31.330062 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568c738-b3cc-4aa9-a34c-d594d3fb571e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:31.330434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568c738-b3cc-4aa9-a34c-d594d3fb571e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:31.332048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568c738-b3cc-4aa9-a34c-d594d3fb571e None admin] RBAC: Adding query filter params (interface=admin, service_id=a08ab7c866184d109df58cb80e31e509, region_id=RegionOne) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:31.332477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568c738-b3cc-4aa9-a34c-d594d3fb571e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:31.333935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568c738-b3cc-4aa9-a34c-d594d3fb571e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:31.343767 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 499/998] 10.222.0.67 () {56 vars in 1249 bytes} [Fri Jun 24 15:01:31 2022] GET /identity/v3/endpoints?service_id=a08ab7c866184d109df58cb80e31e509&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:31.344880 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568c738-b3cc-4aa9-a34c-d594d3fb571e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:31.350620 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:31.361496 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:31.428275 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64739a3-ed03-411f-bd6f-27d1eb870980 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:31.458850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568c738-b3cc-4aa9-a34c-d594d3fb571e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:31.464701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff7095-38e3-4d3e-a7b7-ae80394b3cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:31.465363 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff7095-38e3-4d3e-a7b7-ae80394b3cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:31.465642 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff7095-38e3-4d3e-a7b7-ae80394b3cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:31.465916 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff7095-38e3-4d3e-a7b7-ae80394b3cb5 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:31.466615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff7095-38e3-4d3e-a7b7-ae80394b3cb5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:31.466944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff7095-38e3-4d3e-a7b7-ae80394b3cb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:31.468338 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff7095-38e3-4d3e-a7b7-ae80394b3cb5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:31.473560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 500/999] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:31.478807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff7095-38e3-4d3e-a7b7-ae80394b3cb5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:31.496874 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e297a6-6411-4ea9-97bf-fa191db8d8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:31.497582 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e297a6-6411-4ea9-97bf-fa191db8d8cf None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:31.497869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e297a6-6411-4ea9-97bf-fa191db8d8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:31.498148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e297a6-6411-4ea9-97bf-fa191db8d8cf None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:31.498830 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e297a6-6411-4ea9-97bf-fa191db8d8cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:31.500372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e297a6-6411-4ea9-97bf-fa191db8d8cf None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:31.507831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 500/1000] 10.222.0.67 () {56 vars in 1087 bytes} [Fri Jun 24 15:01:31 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:01:31.588482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:31.599208 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff7095-38e3-4d3e-a7b7-ae80394b3cb5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:31.637617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafab06a-f5c4-4e54-b228-1d97c84e3435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:31.638491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafab06a-f5c4-4e54-b228-1d97c84e3435 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:31.638878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafab06a-f5c4-4e54-b228-1d97c84e3435 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:31.639286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafab06a-f5c4-4e54-b228-1d97c84e3435 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:31.640115 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafab06a-f5c4-4e54-b228-1d97c84e3435 None admin] RBAC: Adding query filter params (type=image) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:31.640526 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafab06a-f5c4-4e54-b228-1d97c84e3435 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:31.642018 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafab06a-f5c4-4e54-b228-1d97c84e3435 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:31.648540 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 501/1001] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:31 2022] GET /identity/v3/services?type=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:32.983969 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cfef2-5f4f-4205-a23d-5eed4a31d908 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:32.984491 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cfef2-5f4f-4205-a23d-5eed4a31d908 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:32.984573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cfef2-5f4f-4205-a23d-5eed4a31d908 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:32.985932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 501/1002] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:32.991129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486c18c-fc11-4998-8949-c9f713c77f6b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:32.991539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486c18c-fc11-4998-8949-c9f713c77f6b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:32.991708 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486c18c-fc11-4998-8949-c9f713c77f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:33.008491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2486c18c-fc11-4998-8949-c9f713c77f6b None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:33.009662 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2486c18c-fc11-4998-8949-c9f713c77f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:33.084873 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 502/1003] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:33.107955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688cfef2-5f4f-4205-a23d-5eed4a31d908 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:33.198213 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:33.222018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688cfef2-5f4f-4205-a23d-5eed4a31d908 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:33.272809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4846b-0cf0-4293-86f5-f0b6b6db222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:33.273535 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4846b-0cf0-4293-86f5-f0b6b6db222d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:33.273817 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4846b-0cf0-4293-86f5-f0b6b6db222d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:33.274091 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4846b-0cf0-4293-86f5-f0b6b6db222d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:33.274745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d4846b-0cf0-4293-86f5-f0b6b6db222d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:33.276166 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d4846b-0cf0-4293-86f5-f0b6b6db222d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-45d4846b-0cf0-4293-86f5-f0b6b6db222d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:33.288014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:33.289072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:33.289985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:33.289985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:33.289985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:33.289985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:33.289985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:33.289985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:33.289985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:33.289985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:33.289985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:33.289985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:33.289985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:33.289985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:33.289985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 15:01:33.289985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:33.294037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 502/1004] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:33.295303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486c18c-fc11-4998-8949-c9f713c77f6b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:33.383328 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:33.399786 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486c18c-fc11-4998-8949-c9f713c77f6b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:33.434945 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383fd47-67c8-4e95-81e0-1a1d550c983b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:33.435575 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383fd47-67c8-4e95-81e0-1a1d550c983b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:33.435862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383fd47-67c8-4e95-81e0-1a1d550c983b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:33.436142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383fd47-67c8-4e95-81e0-1a1d550c983b None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:33.436843 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383fd47-67c8-4e95-81e0-1a1d550c983b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:33.437152 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383fd47-67c8-4e95-81e0-1a1d550c983b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:33.438517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383fd47-67c8-4e95-81e0-1a1d550c983b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:33.448791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 503/1005] 10.222.0.67 () {56 vars in 1112 bytes} [Fri Jun 24 15:01:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:33.450337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4846b-0cf0-4293-86f5-f0b6b6db222d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:33.552019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4846b-0cf0-4293-86f5-f0b6b6db222d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:33.576279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919e909-7960-4dc0-b11e-a0317e971a93 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:33.576592 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919e909-7960-4dc0-b11e-a0317e971a93 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:33.576950 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919e909-7960-4dc0-b11e-a0317e971a93 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:33.578467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 504/1006] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:33.583928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea779fbf-6ed5-4a0a-99e9-e93517cace5f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:33.584214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea779fbf-6ed5-4a0a-99e9-e93517cace5f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:33.584494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea779fbf-6ed5-4a0a-99e9-e93517cace5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:33.586605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012f57b-0286-4e3c-a58f-fe5f5a5c0f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:33.587294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012f57b-0286-4e3c-a58f-fe5f5a5c0f54 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:33.587578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012f57b-0286-4e3c-a58f-fe5f5a5c0f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:33.587853 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012f57b-0286-4e3c-a58f-fe5f5a5c0f54 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:33.588555 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012f57b-0286-4e3c-a58f-fe5f5a5c0f54 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:33.588857 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012f57b-0286-4e3c-a58f-fe5f5a5c0f54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:33.590404 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012f57b-0286-4e3c-a58f-fe5f5a5c0f54 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:33.602555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ea779fbf-6ed5-4a0a-99e9-e93517cace5f None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:33.603520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 503/1007] 10.222.0.67 () {56 vars in 1112 bytes} [Fri Jun 24 15:01:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:33.603775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ea779fbf-6ed5-4a0a-99e9-e93517cace5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:33.606007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012f57b-0286-4e3c-a58f-fe5f5a5c0f54 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:33.610335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:33.673505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 505/1008] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:33.697565 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea779fbf-6ed5-4a0a-99e9-e93517cace5f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:33.715371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012f57b-0286-4e3c-a58f-fe5f5a5c0f54 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:33.751273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fafde89-41ed-4bbe-bf2b-11eb2d6da5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:33.751941 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafde89-41ed-4bbe-bf2b-11eb2d6da5c9 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:33.752230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafde89-41ed-4bbe-bf2b-11eb2d6da5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:33.753183 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafde89-41ed-4bbe-bf2b-11eb2d6da5c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:33.754255 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fafde89-41ed-4bbe-bf2b-11eb2d6da5c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:33.755666 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fafde89-41ed-4bbe-bf2b-11eb2d6da5c9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:33.825847 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 504/1009] 10.222.0.67 () {60 vars in 1144 bytes} [Fri Jun 24 15:01:33 2022] POST /identity/v3/endpoints => generated 334 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:01:33.827130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e6122-8df8-4b5c-a9c4-8931cc0bddc2 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:33.827463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e6122-8df8-4b5c-a9c4-8931cc0bddc2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:33.827693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:33.827809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e6122-8df8-4b5c-a9c4-8931cc0bddc2 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:33.829457 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 505/1010] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:33.841364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d6883-a384-475b-8156-4c608095c78e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:33.841701 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d6883-a384-475b-8156-4c608095c78e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:33.841982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d6883-a384-475b-8156-4c608095c78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:33.854889 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea779fbf-6ed5-4a0a-99e9-e93517cace5f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:33.861182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-244d6883-a384-475b-8156-4c608095c78e None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:33.862653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-244d6883-a384-475b-8156-4c608095c78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:33.904641 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4ec62-6f4a-4e22-ada3-f84239657298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:33.905536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4ec62-6f4a-4e22-ada3-f84239657298 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:33.905830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4ec62-6f4a-4e22-ada3-f84239657298 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:33.906132 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4ec62-6f4a-4e22-ada3-f84239657298 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:33.906776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4ec62-6f4a-4e22-ada3-f84239657298 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:33.908397 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4ec62-6f4a-4e22-ada3-f84239657298 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b8d4ec62-6f4a-4e22-ada3-f84239657298 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:33.923059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:33.924630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:33.926036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:33.926036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:33.926036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:33.926036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:33.926036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:33.926036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:33.926036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:33.926036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:33.926036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:33.926036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:33.926036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:33.926036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:33.926036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 15:01:33.926036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:33.930206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:33.930862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 506/1011] 10.222.0.67 () {56 vars in 1126 bytes} [Fri Jun 24 15:01:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:33.931649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4ec62-6f4a-4e22-ada3-f84239657298 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:33.942484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 506/1012] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:33 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:33.966529 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d6883-a384-475b-8156-4c608095c78e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:34.054238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4ec62-6f4a-4e22-ada3-f84239657298 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:34.089407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d6883-a384-475b-8156-4c608095c78e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:34.100356 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef562963-77a2-4465-9041-6ef07c7dfcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:34.101031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef562963-77a2-4465-9041-6ef07c7dfcfc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:34.101317 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef562963-77a2-4465-9041-6ef07c7dfcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:34.101595 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef562963-77a2-4465-9041-6ef07c7dfcfc None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:34.102309 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef562963-77a2-4465-9041-6ef07c7dfcfc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:34.102618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef562963-77a2-4465-9041-6ef07c7dfcfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:34.104085 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef562963-77a2-4465-9041-6ef07c7dfcfc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:34.109360 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 507/1013] 10.222.0.67 () {56 vars in 1122 bytes} [Fri Jun 24 15:01:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:34.115458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef562963-77a2-4465-9041-6ef07c7dfcfc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:34.118209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:34.134129 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c571635-c6c8-4273-935f-e3a9992d5b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:34.136235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c571635-c6c8-4273-935f-e3a9992d5b30 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:34.136235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c571635-c6c8-4273-935f-e3a9992d5b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:34.136235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c571635-c6c8-4273-935f-e3a9992d5b30 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:34.136580 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c571635-c6c8-4273-935f-e3a9992d5b30 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:34.138435 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c571635-c6c8-4273-935f-e3a9992d5b30 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:34.207418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 507/1014] 10.222.0.67 () {60 vars in 1141 bytes} [Fri Jun 24 15:01:33 2022] POST /identity/v3/services => generated 242 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:01:34.230351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef562963-77a2-4465-9041-6ef07c7dfcfc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:34.275667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab09fe-9c57-478b-9f62-ae7ca81acd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:34.276349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab09fe-9c57-478b-9f62-ae7ca81acd79 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:34.276631 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab09fe-9c57-478b-9f62-ae7ca81acd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:34.276904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab09fe-9c57-478b-9f62-ae7ca81acd79 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:34.277618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab09fe-9c57-478b-9f62-ae7ca81acd79 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:34.277926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab09fe-9c57-478b-9f62-ae7ca81acd79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:34.279344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab09fe-9c57-478b-9f62-ae7ca81acd79 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:34.290069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 508/1015] 10.222.0.67 () {56 vars in 1122 bytes} [Fri Jun 24 15:01:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:34.291407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c571635-c6c8-4273-935f-e3a9992d5b30 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:34.345829 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:34.399264 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c571635-c6c8-4273-935f-e3a9992d5b30 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:34.436028 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7624604-4236-4f2c-a7e3-9b5e8a9b16ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:34.436702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7624604-4236-4f2c-a7e3-9b5e8a9b16ae None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:34.436988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7624604-4236-4f2c-a7e3-9b5e8a9b16ae None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:34.437270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7624604-4236-4f2c-a7e3-9b5e8a9b16ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:34.438017 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7624604-4236-4f2c-a7e3-9b5e8a9b16ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:34.439415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7624604-4236-4f2c-a7e3-9b5e8a9b16ae None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:34.495398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 508/1016] 10.222.0.67 () {60 vars in 1144 bytes} [Fri Jun 24 15:01:34 2022] POST /identity/v3/endpoints => generated 325 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:01:35.964369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56cb33-b197-44ab-9b82-b251b767402c None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:35.964904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56cb33-b197-44ab-9b82-b251b767402c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:35.964904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56cb33-b197-44ab-9b82-b251b767402c None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:35.966316 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 509/1017] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:35.971854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67362270-9cd4-4d12-8d29-9053f8181afe None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:35.972262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67362270-9cd4-4d12-8d29-9053f8181afe None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:35.972467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67362270-9cd4-4d12-8d29-9053f8181afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:35.988907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-67362270-9cd4-4d12-8d29-9053f8181afe None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:35.990073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-67362270-9cd4-4d12-8d29-9053f8181afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:36.064492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 509/1018] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:36.090613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56cb33-b197-44ab-9b82-b251b767402c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:36.157181 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:36.199021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56cb33-b197-44ab-9b82-b251b767402c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:36.232633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fd33e-ee64-4570-92ff-01f0bd565c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:36.233379 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fd33e-ee64-4570-92ff-01f0bd565c76 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:36.233660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fd33e-ee64-4570-92ff-01f0bd565c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:36.233935 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fd33e-ee64-4570-92ff-01f0bd565c76 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:36.244766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047fd33e-ee64-4570-92ff-01f0bd565c76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:01:36.251556 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2d90-d025-4b4b-b09c-b2b2124db226 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:36.251851 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2d90-d025-4b4b-b09c-b2b2124db226 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:36.252134 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2d90-d025-4b4b-b09c-b2b2124db226 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-047fd33e-ee64-4570-92ff-01f0bd565c76 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:01:36.253105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:36.254024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:36.254872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:36.254872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:36.254872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:36.254872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:36.254872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:01:36.254872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:01:36.254872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:01:36.254872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:01:36.254872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:01:36.254872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:01:36.254872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 15:01:36.254872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:36.255553 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 510/1019] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:36.255625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 510/1020] 10.222.0.67 () {56 vars in 1120 bytes} [Fri Jun 24 15:01:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:36.258668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3942ff-739a-4474-b134-1d3bd3bd0743 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:36.258959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3942ff-739a-4474-b134-1d3bd3bd0743 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:36.259354 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3942ff-739a-4474-b134-1d3bd3bd0743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:36.260174 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fd33e-ee64-4570-92ff-01f0bd565c76 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:36.276325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2c3942ff-739a-4474-b134-1d3bd3bd0743 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:36.277515 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2c3942ff-739a-4474-b134-1d3bd3bd0743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:36.310134 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:36.355266 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 511/1021] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:36.364963 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fd33e-ee64-4570-92ff-01f0bd565c76 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:36.376258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3942ff-739a-4474-b134-1d3bd3bd0743 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:36.389416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:36.401396 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d3ffb-0ca4-44fd-abd0-a071bede8eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:36.402063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d3ffb-0ca4-44fd-abd0-a071bede8eea None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:36.402343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d3ffb-0ca4-44fd-abd0-a071bede8eea None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:36.402747 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d3ffb-0ca4-44fd-abd0-a071bede8eea None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:36.403382 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d3ffb-0ca4-44fd-abd0-a071bede8eea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:36.403691 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d3ffb-0ca4-44fd-abd0-a071bede8eea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:36.405143 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d3ffb-0ca4-44fd-abd0-a071bede8eea None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:36.418618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 511/1022] 10.222.0.67 () {56 vars in 1115 bytes} [Fri Jun 24 15:01:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:36.420194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d3ffb-0ca4-44fd-abd0-a071bede8eea None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:36.483915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3942ff-739a-4474-b134-1d3bd3bd0743 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:36.519861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9b621-4c6e-4fd6-abd9-a048d88c7d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:36.520651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9b621-4c6e-4fd6-abd9-a048d88c7d78 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:36.520930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9b621-4c6e-4fd6-abd9-a048d88c7d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:36.521287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9b621-4c6e-4fd6-abd9-a048d88c7d78 None admin] PATH_INFO: `/v3/services/image` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:36.521840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9b621-4c6e-4fd6-abd9-a048d88c7d78 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:36.523214 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9b621-4c6e-4fd6-abd9-a048d88c7d78 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:36.529363 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d3ffb-0ca4-44fd-abd0-a071bede8eea None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-76b9b621-4c6e-4fd6-abd9-a048d88c7d78 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:36.535759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:36.536745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:36.537815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:36.537815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:36.537815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:36.537815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:36.537815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:36.537815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:36.537815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:36.537815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:36.537815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:36.537815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:36.537815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:36.537815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:36.537815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 15:01:36.537815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:36.537815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 512/1023] 10.222.0.67 () {56 vars in 1105 bytes} [Fri Jun 24 15:01:36 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:36.542833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9b621-4c6e-4fd6-abd9-a048d88c7d78 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:36.570802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c67db-c1de-4057-8705-ce14550097f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:36.571635 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c67db-c1de-4057-8705-ce14550097f2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:36.571927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c67db-c1de-4057-8705-ce14550097f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:36.572206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c67db-c1de-4057-8705-ce14550097f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:36.574888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:36.582646 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c67db-c1de-4057-8705-ce14550097f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:36.584131 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c67db-c1de-4057-8705-ce14550097f2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:36.601874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-564c67db-c1de-4057-8705-ce14550097f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:36.601874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:36.601874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:36.601874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:36.601874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:36.601874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:36.601874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:36.601874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:36.601874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:36.601874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:36.601874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:36.601874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:36.601874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:36.601874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:36.601874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:36.602823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:36.603762 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:36.603762 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:36.603762 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:36.603762 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:36.603762 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 512/1024] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:36.604724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3ace4-444e-4fab-a38b-2844800714eb None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:36.605007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3ace4-444e-4fab-a38b-2844800714eb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:36.605283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3ace4-444e-4fab-a38b-2844800714eb None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:36.606552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 513/1025] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:36.609188 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3ace4-444e-4fab-a38b-2844800714eb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:36.646846 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:36.656547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9b621-4c6e-4fd6-abd9-a048d88c7d78 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:36.691678 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a8af4-b9f4-4e58-aa00-02b179b83ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:36.692480 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a8af4-b9f4-4e58-aa00-02b179b83ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:36.692852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a8af4-b9f4-4e58-aa00-02b179b83ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:36.693217 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a8af4-b9f4-4e58-aa00-02b179b83ec4 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:36.694040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a8af4-b9f4-4e58-aa00-02b179b83ec4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:36.694434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a8af4-b9f4-4e58-aa00-02b179b83ec4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:36.695949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a8af4-b9f4-4e58-aa00-02b179b83ec4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:36.700799 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 513/1026] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:36.702289 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93ae97-4f50-4b7e-a6bc-838443d01e9a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:36.702485 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93ae97-4f50-4b7e-a6bc-838443d01e9a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:36.702764 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93ae97-4f50-4b7e-a6bc-838443d01e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:36.711675 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3ace4-444e-4fab-a38b-2844800714eb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:36.718796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3a93ae97-4f50-4b7e-a6bc-838443d01e9a None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:36.720001 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3a93ae97-4f50-4b7e-a6bc-838443d01e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:36.746771 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df96484-2676-4e24-a0e2-9770f87575bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:36.747401 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df96484-2676-4e24-a0e2-9770f87575bb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:36.747684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df96484-2676-4e24-a0e2-9770f87575bb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:36.748018 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df96484-2676-4e24-a0e2-9770f87575bb None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:36.748734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df96484-2676-4e24-a0e2-9770f87575bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:36.749037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df96484-2676-4e24-a0e2-9770f87575bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:36.750441 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df96484-2676-4e24-a0e2-9770f87575bb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:36.761622 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 514/1027] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:36.763790 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df96484-2676-4e24-a0e2-9770f87575bb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:36.787939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 514/1028] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:36.788768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93ae97-4f50-4b7e-a6bc-838443d01e9a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:36.824304 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:36.860427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:36.867012 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df96484-2676-4e24-a0e2-9770f87575bb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:36.899829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2765a28-0828-4390-8e90-093d3f5ab1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:36.900501 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93ae97-4f50-4b7e-a6bc-838443d01e9a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:36.900608 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2765a28-0828-4390-8e90-093d3f5ab1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:36.900785 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2765a28-0828-4390-8e90-093d3f5ab1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:36.901064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2765a28-0828-4390-8e90-093d3f5ab1c5 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:36.901764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2765a28-0828-4390-8e90-093d3f5ab1c5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:36.902073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2765a28-0828-4390-8e90-093d3f5ab1c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:36.903488 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2765a28-0828-4390-8e90-093d3f5ab1c5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:36.914300 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 515/1029] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:36.914300 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2765a28-0828-4390-8e90-093d3f5ab1c5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:36.940444 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b84db-9fa0-4915-9be1-c682a7fc707c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:36.941241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b84db-9fa0-4915-9be1-c682a7fc707c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:36.941524 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b84db-9fa0-4915-9be1-c682a7fc707c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:36.941801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b84db-9fa0-4915-9be1-c682a7fc707c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:36.949015 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b84db-9fa0-4915-9be1-c682a7fc707c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:36.950447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b84db-9fa0-4915-9be1-c682a7fc707c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-2c8b84db-9fa0-4915-9be1-c682a7fc707c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:01:36.964821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:36.965825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:36.966852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:36.966852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:36.966852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:01:36.966852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:01:36.966852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:01:36.966852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:01:36.966852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:01:36.966852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:01:36.966852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:01:36.966852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:36.966852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 515/1030] 10.222.0.67 () {56 vars in 1093 bytes} [Fri Jun 24 15:01:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:36.966852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b84db-9fa0-4915-9be1-c682a7fc707c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:37.012448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2765a28-0828-4390-8e90-093d3f5ab1c5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:37.045805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c694382-5bd2-4719-947c-f8b4f2aaf69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:37.046600 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c694382-5bd2-4719-947c-f8b4f2aaf69d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:37.046887 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c694382-5bd2-4719-947c-f8b4f2aaf69d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:37.047167 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c694382-5bd2-4719-947c-f8b4f2aaf69d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:37.048368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c694382-5bd2-4719-947c-f8b4f2aaf69d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:37.049839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c694382-5bd2-4719-947c-f8b4f2aaf69d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:37.051706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:37.052135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 516/1031] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:37.052509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c694382-5bd2-4719-947c-f8b4f2aaf69d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:37.072524 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b84db-9fa0-4915-9be1-c682a7fc707c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:37.079614 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:37.110523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6605a02-d457-4d57-b1c6-018c3bce7164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:37.111232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6605a02-d457-4d57-b1c6-018c3bce7164 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:37.111528 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6605a02-d457-4d57-b1c6-018c3bce7164 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:37.111875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6605a02-d457-4d57-b1c6-018c3bce7164 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:37.112709 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6605a02-d457-4d57-b1c6-018c3bce7164 None admin] RBAC: Adding query filter params (interface=public, service_id=3f4c776e2f194440aecdcb84856eadac, region_id=RegionOne) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:37.113041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6605a02-d457-4d57-b1c6-018c3bce7164 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:37.114393 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6605a02-d457-4d57-b1c6-018c3bce7164 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:37.119380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 516/1032] 10.222.0.67 () {56 vars in 1251 bytes} [Fri Jun 24 15:01:36 2022] GET /identity/v3/endpoints?service_id=3f4c776e2f194440aecdcb84856eadac&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:37.120116 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6605a02-d457-4d57-b1c6-018c3bce7164 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:37.154404 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c694382-5bd2-4719-947c-f8b4f2aaf69d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:37.188947 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c3e45-0312-4af6-a12f-a58e8e1b33de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:37.189640 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c3e45-0312-4af6-a12f-a58e8e1b33de None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:37.190039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c3e45-0312-4af6-a12f-a58e8e1b33de None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:37.190322 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c3e45-0312-4af6-a12f-a58e8e1b33de None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:37.191147 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c3e45-0312-4af6-a12f-a58e8e1b33de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:37.191521 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c3e45-0312-4af6-a12f-a58e8e1b33de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:37.192944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c3e45-0312-4af6-a12f-a58e8e1b33de None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:37.214986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 517/1033] 10.222.0.67 () {56 vars in 1133 bytes} [Fri Jun 24 15:01:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:37.214986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c3e45-0312-4af6-a12f-a58e8e1b33de None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:37.217659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6605a02-d457-4d57-b1c6-018c3bce7164 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:37.252150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2009be-689c-4133-9bd6-1510c55064e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:37.252800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2009be-689c-4133-9bd6-1510c55064e8 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:37.253086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2009be-689c-4133-9bd6-1510c55064e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:37.253357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2009be-689c-4133-9bd6-1510c55064e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:37.254056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2009be-689c-4133-9bd6-1510c55064e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:37.255622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2009be-689c-4133-9bd6-1510c55064e8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:37.293627 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:37.308051 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c3e45-0312-4af6-a12f-a58e8e1b33de None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:37.324011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:37.331071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 517/1034] 10.222.0.67 () {60 vars in 1141 bytes} [Fri Jun 24 15:01:37 2022] POST /identity/v3/projects => generated 311 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:01:37.331738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2009be-689c-4133-9bd6-1510c55064e8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:37.348971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53213e-b3b2-428d-900e-1f4fa7a78ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:37.350033 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53213e-b3b2-428d-900e-1f4fa7a78ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:37.350485 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53213e-b3b2-428d-900e-1f4fa7a78ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:37.350926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53213e-b3b2-428d-900e-1f4fa7a78ca8 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:37.351818 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a53213e-b3b2-428d-900e-1f4fa7a78ca8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:37.353475 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a53213e-b3b2-428d-900e-1f4fa7a78ca8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:37.371212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 518/1035] 10.222.0.67 () {56 vars in 1087 bytes} [Fri Jun 24 15:01:37 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:01:37.453238 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2009be-689c-4133-9bd6-1510c55064e8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:37.493248 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573eae1-04ba-481a-be5a-35f60e86ed39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:37.493925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573eae1-04ba-481a-be5a-35f60e86ed39 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:37.494210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573eae1-04ba-481a-be5a-35f60e86ed39 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:37.494487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573eae1-04ba-481a-be5a-35f60e86ed39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:37.565588 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:37.567993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573eae1-04ba-481a-be5a-35f60e86ed39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:37.569455 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573eae1-04ba-481a-be5a-35f60e86ed39 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:37.668945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 518/1036] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:01:37 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:01:37.674791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53213e-b3b2-428d-900e-1f4fa7a78ca8 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:37.778211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53213e-b3b2-428d-900e-1f4fa7a78ca8 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:37.813362 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b5f5c-5b0a-4d6c-9c8d-9d3f459d3792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:37.814161 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b5f5c-5b0a-4d6c-9c8d-9d3f459d3792 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:37.814448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b5f5c-5b0a-4d6c-9c8d-9d3f459d3792 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:37.814898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b5f5c-5b0a-4d6c-9c8d-9d3f459d3792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:37.825391 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b5f5c-5b0a-4d6c-9c8d-9d3f459d3792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:37.826782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b5f5c-5b0a-4d6c-9c8d-9d3f459d3792 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:37.845170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-318b5f5c-5b0a-4d6c-9c8d-9d3f459d3792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:37.845170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:37.845170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:37.845170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:37.845170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:37.845170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:37.845170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:37.845170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:37.845170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:37.845170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:37.845170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:37.845170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:37.845170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:37.845170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:37.845170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:37.846687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:37.847984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:37.847984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:37.847984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:37.847984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:37.851602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 519/1037] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:37.852846 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573eae1-04ba-481a-be5a-35f60e86ed39 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:37.949652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:37.952862 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573eae1-04ba-481a-be5a-35f60e86ed39 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:37.987249 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66277e-9b9b-42b1-9a7d-57d0a64b7d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:37.987910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66277e-9b9b-42b1-9a7d-57d0a64b7d9c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:37.988196 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66277e-9b9b-42b1-9a7d-57d0a64b7d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:37.988479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66277e-9b9b-42b1-9a7d-57d0a64b7d9c None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:37.989197 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66277e-9b9b-42b1-9a7d-57d0a64b7d9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:37.989504 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66277e-9b9b-42b1-9a7d-57d0a64b7d9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:37.990853 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66277e-9b9b-42b1-9a7d-57d0a64b7d9c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:38.006476 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 519/1038] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:38.007862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b5f5c-5b0a-4d6c-9c8d-9d3f459d3792 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:38.010710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:38.116496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b5f5c-5b0a-4d6c-9c8d-9d3f459d3792 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:38.151620 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1972c47-8e24-44a6-a639-1a7664388b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:38.152418 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1972c47-8e24-44a6-a639-1a7664388b71 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:38.152698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1972c47-8e24-44a6-a639-1a7664388b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:38.152968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1972c47-8e24-44a6-a639-1a7664388b71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:38.159839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1972c47-8e24-44a6-a639-1a7664388b71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:38.161283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1972c47-8e24-44a6-a639-1a7664388b71 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:38.175351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d1972c47-8e24-44a6-a639-1a7664388b71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:38.175351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:38.175351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:38.175351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:38.175351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:38.175351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:38.175351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:38.175351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:38.175351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:38.175351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:38.175351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:38.175351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:38.175351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:01:38.175351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:01:38.175351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:38.176230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:38.177447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:01:38.177447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:01:38.177447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:01:38.177447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:01:38.177447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:01:38.177447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:01:38.177447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:38.177447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:38.178811 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 520/1039] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:38.182483 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66277e-9b9b-42b1-9a7d-57d0a64b7d9c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:38.281349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66277e-9b9b-42b1-9a7d-57d0a64b7d9c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:38.313986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4da337-bbe7-4003-a442-80f19feec92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:38.315161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4da337-bbe7-4003-a442-80f19feec92f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:38.315668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4da337-bbe7-4003-a442-80f19feec92f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:38.315942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4da337-bbe7-4003-a442-80f19feec92f None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:38.316723 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4da337-bbe7-4003-a442-80f19feec92f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:38.317028 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4da337-bbe7-4003-a442-80f19feec92f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:38.318445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4da337-bbe7-4003-a442-80f19feec92f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:38.331685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 520/1040] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:01:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:38.337366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1972c47-8e24-44a6-a639-1a7664388b71 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:38.400596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:38.439852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1972c47-8e24-44a6-a639-1a7664388b71 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:38.477692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4827f1-805b-48c6-b2a3-08343b448f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:38.478354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4827f1-805b-48c6-b2a3-08343b448f48 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:38.478634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4827f1-805b-48c6-b2a3-08343b448f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:38.478907 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4827f1-805b-48c6-b2a3-08343b448f48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:38.479773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4827f1-805b-48c6-b2a3-08343b448f48 None admin] RBAC: Adding query filter params (role.id=f70de858079b47cf92a3995fda64bbc0, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=51de59caa3144a1cbf12b20b15e2134f) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:38.480089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4827f1-805b-48c6-b2a3-08343b448f48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:38.481517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4827f1-805b-48c6-b2a3-08343b448f48 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:38.488954 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 521/1041] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:01:38 2022] GET /identity/v3/role_assignments?user.id=51de59caa3144a1cbf12b20b15e2134f&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=f70de858079b47cf92a3995fda64bbc0 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:39.430811 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f2c6c-97a1-44af-8ee8-d9143d957304 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:39.431385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f2c6c-97a1-44af-8ee8-d9143d957304 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:39.431385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f2c6c-97a1-44af-8ee8-d9143d957304 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:39.432899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 521/1042] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:39.437797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837aa680-7127-495d-9ad3-7ed6feac597b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:39.438178 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837aa680-7127-495d-9ad3-7ed6feac597b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:39.438355 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837aa680-7127-495d-9ad3-7ed6feac597b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:39.455367 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-837aa680-7127-495d-9ad3-7ed6feac597b None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:39.456543 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-837aa680-7127-495d-9ad3-7ed6feac597b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:39.466641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9667c17-00c9-4a51-9d1c-51583d8dd64d None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:39.466974 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9667c17-00c9-4a51-9d1c-51583d8dd64d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:39.467283 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9667c17-00c9-4a51-9d1c-51583d8dd64d None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:39.468762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 522/1043] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:01:39.474404 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ccaac-95f1-41d7-805c-c6c575d04c4d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:39.474537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ccaac-95f1-41d7-805c-c6c575d04c4d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:39.474819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ccaac-95f1-41d7-805c-c6c575d04c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:39.491594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-976ccaac-95f1-41d7-805c-c6c575d04c4d None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:39.493428 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-976ccaac-95f1-41d7-805c-c6c575d04c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:39.510958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:39.526050 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 522/1044] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:39.549737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837aa680-7127-495d-9ad3-7ed6feac597b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:39.573659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 523/1045] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:39.599698 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ccaac-95f1-41d7-805c-c6c575d04c4d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:39.612042 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:39.672811 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837aa680-7127-495d-9ad3-7ed6feac597b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:39.711207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ccaac-95f1-41d7-805c-c6c575d04c4d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:39.718021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5dee8-76d6-41e9-a7e6-3efa5dc2d003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:39.719247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5dee8-76d6-41e9-a7e6-3efa5dc2d003 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:39.719542 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5dee8-76d6-41e9-a7e6-3efa5dc2d003 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:39.719822 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5dee8-76d6-41e9-a7e6-3efa5dc2d003 None admin] PATH_INFO: `/v3/services/image` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:39.721349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5dee8-76d6-41e9-a7e6-3efa5dc2d003 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:39.723045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5dee8-76d6-41e9-a7e6-3efa5dc2d003 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-54b5dee8-76d6-41e9-a7e6-3efa5dc2d003 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:39.735416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:39.737129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:39.744294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:39.744294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:39.744294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:39.744294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:39.744294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:39.744294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:39.744294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:39.744294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:39.744294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:39.744294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:39.744294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:01:39.744294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:01:39.744294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 15:01:39.744294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:39.744294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 523/1046] 10.222.0.67 () {56 vars in 1105 bytes} [Fri Jun 24 15:01:39 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:39.745987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5dee8-76d6-41e9-a7e6-3efa5dc2d003 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:39.752695 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee612c-2949-4300-957b-ea773b11a9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:39.753515 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee612c-2949-4300-957b-ea773b11a9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:39.753796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee612c-2949-4300-957b-ea773b11a9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:39.754069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee612c-2949-4300-957b-ea773b11a9d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:39.755364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee612c-2949-4300-957b-ea773b11a9d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:39.756896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee612c-2949-4300-957b-ea773b11a9d3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:39.758716 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 524/1047] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:39.766262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee612c-2949-4300-957b-ea773b11a9d3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:39.790653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:39.870875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5dee8-76d6-41e9-a7e6-3efa5dc2d003 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:39.882850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee612c-2949-4300-957b-ea773b11a9d3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:39.893488 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:39.909669 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2def94-9f14-4d04-bef1-8e26590153d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:39.910451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2def94-9f14-4d04-bef1-8e26590153d5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:39.910741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2def94-9f14-4d04-bef1-8e26590153d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:39.911032 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2def94-9f14-4d04-bef1-8e26590153d5 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:39.912084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2def94-9f14-4d04-bef1-8e26590153d5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:39.912404 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2def94-9f14-4d04-bef1-8e26590153d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:39.914155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2def94-9f14-4d04-bef1-8e26590153d5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:39.919549 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 524/1048] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:39.923144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14870e3a-fbf4-445d-ad08-a00f521c9260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:39.924857 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14870e3a-fbf4-445d-ad08-a00f521c9260 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:39.925142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14870e3a-fbf4-445d-ad08-a00f521c9260 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:39.925415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14870e3a-fbf4-445d-ad08-a00f521c9260 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:39.925663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2def94-9f14-4d04-bef1-8e26590153d5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:39.926573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14870e3a-fbf4-445d-ad08-a00f521c9260 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:39.929231 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14870e3a-fbf4-445d-ad08-a00f521c9260 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:40.038158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2def94-9f14-4d04-bef1-8e26590153d5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:40.051714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 525/1049] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 15:01:39 2022] POST /identity/v3/users => generated 285 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:01:40.101766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2274de-d7a5-4a69-b602-7b8f183610f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:40.102536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2274de-d7a5-4a69-b602-7b8f183610f8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:40.102894 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2274de-d7a5-4a69-b602-7b8f183610f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:40.103287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2274de-d7a5-4a69-b602-7b8f183610f8 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:40.104114 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2274de-d7a5-4a69-b602-7b8f183610f8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:40.104493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2274de-d7a5-4a69-b602-7b8f183610f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:40.106048 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2274de-d7a5-4a69-b602-7b8f183610f8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:40.117510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 525/1050] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:40.118904 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14870e3a-fbf4-445d-ad08-a00f521c9260 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:40.210071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:40.241835 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14870e3a-fbf4-445d-ad08-a00f521c9260 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:40.286097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea2d1b-03ee-4fb0-8678-8a7b33708015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:40.286778 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea2d1b-03ee-4fb0-8678-8a7b33708015 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:40.287065 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea2d1b-03ee-4fb0-8678-8a7b33708015 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:40.287373 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea2d1b-03ee-4fb0-8678-8a7b33708015 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:40.288128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea2d1b-03ee-4fb0-8678-8a7b33708015 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:40.289570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea2d1b-03ee-4fb0-8678-8a7b33708015 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:40.386428 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 526/1051] 10.222.0.67 () {60 vars in 1144 bytes} [Fri Jun 24 15:01:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:01:40.548560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db26c2-0a2c-449b-a627-06b1461eb18c None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:40.549195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db26c2-0a2c-449b-a627-06b1461eb18c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:40.549195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db26c2-0a2c-449b-a627-06b1461eb18c None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:40.550802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 526/1052] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:40.556475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cee76-be7b-42b7-a95a-1a2c85521c36 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:40.556916 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cee76-be7b-42b7-a95a-1a2c85521c36 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:40.557108 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cee76-be7b-42b7-a95a-1a2c85521c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:40.575850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-658cee76-be7b-42b7-a95a-1a2c85521c36 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:40.577047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-658cee76-be7b-42b7-a95a-1a2c85521c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:40.654630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 527/1053] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:40.677414 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db26c2-0a2c-449b-a627-06b1461eb18c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:40.732529 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:40.786863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db26c2-0a2c-449b-a627-06b1461eb18c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:40.823216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cf732-5a75-44e9-8d30-d061aef61741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:40.823997 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cf732-5a75-44e9-8d30-d061aef61741 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:40.824280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cf732-5a75-44e9-8d30-d061aef61741 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:40.824551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cf732-5a75-44e9-8d30-d061aef61741 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:40.835489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0cf732-5a75-44e9-8d30-d061aef61741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-cb0cf732-5a75-44e9-8d30-d061aef61741 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:01:40.843726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:40.844760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:40.845612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:40.845612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:40.845612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:40.845612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:40.845612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:01:40.845612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:01:40.845612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:01:40.845612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:01:40.845612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:01:40.845612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:01:40.845612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 15:01:40.845612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:40.847075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 527/1054] 10.222.0.67 () {56 vars in 1120 bytes} [Fri Jun 24 15:01:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:40.851214 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cee76-be7b-42b7-a95a-1a2c85521c36 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:40.912880 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:40.957654 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cee76-be7b-42b7-a95a-1a2c85521c36 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:40.992582 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa94876-0007-4ba5-af3e-9cb0c257fcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:40.993250 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa94876-0007-4ba5-af3e-9cb0c257fcbc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:40.993532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa94876-0007-4ba5-af3e-9cb0c257fcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:40.993808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa94876-0007-4ba5-af3e-9cb0c257fcbc None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:40.994555 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa94876-0007-4ba5-af3e-9cb0c257fcbc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:40.994864 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa94876-0007-4ba5-af3e-9cb0c257fcbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:40.996273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa94876-0007-4ba5-af3e-9cb0c257fcbc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:41.004810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 528/1055] 10.222.0.67 () {56 vars in 1115 bytes} [Fri Jun 24 15:01:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:41.010227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cf732-5a75-44e9-8d30-d061aef61741 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:41.113320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cf732-5a75-44e9-8d30-d061aef61741 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:41.122589 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:41.148223 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb5f50-b229-40b7-a34d-073ebd9d5cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:41.149102 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb5f50-b229-40b7-a34d-073ebd9d5cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:41.149364 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb5f50-b229-40b7-a34d-073ebd9d5cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:41.149636 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb5f50-b229-40b7-a34d-073ebd9d5cd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:41.163283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb5f50-b229-40b7-a34d-073ebd9d5cd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:41.164726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb5f50-b229-40b7-a34d-073ebd9d5cd3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:41.181957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-15bb5f50-b229-40b7-a34d-073ebd9d5cd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:41.181957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:41.181957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:41.181957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:41.181957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:41.181957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:41.181957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:41.181957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:41.181957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:41.181957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:41.181957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:41.181957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:41.181957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:41.181957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:41.181957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:41.182892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:41.183867 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:41.183867 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:41.183867 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:41.183867 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:41.183867 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 528/1056] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:41.190588 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa94876-0007-4ba5-af3e-9cb0c257fcbc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:41.289964 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa94876-0007-4ba5-af3e-9cb0c257fcbc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:41.308145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:41.326305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923fbce-7016-41f9-baf9-ca020ae1d6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:41.326974 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923fbce-7016-41f9-baf9-ca020ae1d6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:41.330245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923fbce-7016-41f9-baf9-ca020ae1d6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:41.330245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923fbce-7016-41f9-baf9-ca020ae1d6b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:41.330245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923fbce-7016-41f9-baf9-ca020ae1d6b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:41.330245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923fbce-7016-41f9-baf9-ca020ae1d6b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:41.330245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923fbce-7016-41f9-baf9-ca020ae1d6b9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:41.346790 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 529/1057] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:41.347933 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb5f50-b229-40b7-a34d-073ebd9d5cd3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:41.448567 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb5f50-b229-40b7-a34d-073ebd9d5cd3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:41.481448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2b930-51f1-4818-88c2-07a91badf927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:41.482239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2b930-51f1-4818-88c2-07a91badf927 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:41.482524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2b930-51f1-4818-88c2-07a91badf927 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:41.482800 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2b930-51f1-4818-88c2-07a91badf927 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:41.490728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2b930-51f1-4818-88c2-07a91badf927 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:41.492202 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2b930-51f1-4818-88c2-07a91badf927 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-9cb2b930-51f1-4818-88c2-07a91badf927 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:01:41.506920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:41.507985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:41.509089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:41.509089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:41.509089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:01:41.509089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:01:41.509089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:01:41.509089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:01:41.509089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:01:41.509089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:01:41.509089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:01:41.509089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:41.509089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 529/1058] 10.222.0.67 () {56 vars in 1093 bytes} [Fri Jun 24 15:01:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:41.513946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923fbce-7016-41f9-baf9-ca020ae1d6b9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:41.618031 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923fbce-7016-41f9-baf9-ca020ae1d6b9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:41.652616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e3117-0922-461e-b8a8-f0abfc6740f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:41.653291 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e3117-0922-461e-b8a8-f0abfc6740f0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:41.653575 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e3117-0922-461e-b8a8-f0abfc6740f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:41.653853 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e3117-0922-461e-b8a8-f0abfc6740f0 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:41.654691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e3117-0922-461e-b8a8-f0abfc6740f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:41.655000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e3117-0922-461e-b8a8-f0abfc6740f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:41.656516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e3117-0922-461e-b8a8-f0abfc6740f0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:41.679221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 530/1059] 10.222.0.67 () {56 vars in 1133 bytes} [Fri Jun 24 15:01:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:41.680822 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2b930-51f1-4818-88c2-07a91badf927 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:41.703916 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:41.784433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2b930-51f1-4818-88c2-07a91badf927 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:41.820907 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89f361-a784-4026-bf65-a22118c7a12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:41.821707 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89f361-a784-4026-bf65-a22118c7a12d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:41.822067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89f361-a784-4026-bf65-a22118c7a12d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:41.822343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89f361-a784-4026-bf65-a22118c7a12d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:41.833528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89f361-a784-4026-bf65-a22118c7a12d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:41.836414 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89f361-a784-4026-bf65-a22118c7a12d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:41.853854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-6e89f361-a784-4026-bf65-a22118c7a12d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:41.853854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:41.853854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:41.853854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:41.853854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:41.853854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:41.853854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:41.853854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:41.853854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:41.853854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:41.853854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:41.853854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:41.853854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:41.853854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:41.853854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:41.854772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:41.860389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:41.860389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:41.860389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:41.860389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:41.860389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 530/1060] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:41.863087 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e3117-0922-461e-b8a8-f0abfc6740f0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:41.897935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:41.965735 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e3117-0922-461e-b8a8-f0abfc6740f0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:42.001056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d7064-4eca-47ca-a857-0294b99a15af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:42.001730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d7064-4eca-47ca-a857-0294b99a15af None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:42.002051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d7064-4eca-47ca-a857-0294b99a15af None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:42.002331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d7064-4eca-47ca-a857-0294b99a15af None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:42.003055 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d7064-4eca-47ca-a857-0294b99a15af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:42.003400 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d7064-4eca-47ca-a857-0294b99a15af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:42.004763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d7064-4eca-47ca-a857-0294b99a15af None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:42.021081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 531/1061] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:42.022604 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89f361-a784-4026-bf65-a22118c7a12d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:42.098002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:42.127332 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89f361-a784-4026-bf65-a22118c7a12d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:42.162871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937b346-4b99-4c08-832a-e66386b86872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:42.163623 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937b346-4b99-4c08-832a-e66386b86872 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:42.163910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937b346-4b99-4c08-832a-e66386b86872 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:42.164189 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937b346-4b99-4c08-832a-e66386b86872 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:42.171421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937b346-4b99-4c08-832a-e66386b86872 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:42.172868 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937b346-4b99-4c08-832a-e66386b86872 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:42.187476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f937b346-4b99-4c08-832a-e66386b86872 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:42.187476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:42.187476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:42.187476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:42.187476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:42.187476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:42.187476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:42.187476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:42.187476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:42.187476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:42.187476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:42.187476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:42.187476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:01:42.187476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:01:42.187476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:42.188423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:42.189346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:01:42.189346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:01:42.189346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:01:42.189346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:01:42.189346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:01:42.189346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:01:42.189346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:42.189346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:42.193535 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 531/1062] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:42.194962 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d7064-4eca-47ca-a857-0294b99a15af None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:42.234649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9e0a0-4a80-493a-b8c4-6153e1563fee None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:42.234982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9e0a0-4a80-493a-b8c4-6153e1563fee None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:42.235228 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9e0a0-4a80-493a-b8c4-6153e1563fee None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:42.237783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 532/1063] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:01:42.243199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f60604-298f-4026-9a01-cfcc8c7776e1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:42.243484 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f60604-298f-4026-9a01-cfcc8c7776e1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:42.243857 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f60604-298f-4026-9a01-cfcc8c7776e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:42.260766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-08f60604-298f-4026-9a01-cfcc8c7776e1 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:42.261946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-08f60604-298f-4026-9a01-cfcc8c7776e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:42.296623 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:42.303169 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d7064-4eca-47ca-a857-0294b99a15af None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:42.338561 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2e970-d6c8-4699-9e9c-857095a2664d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:42.339255 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2e970-d6c8-4699-9e9c-857095a2664d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:42.339545 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2e970-d6c8-4699-9e9c-857095a2664d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:42.339868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2e970-d6c8-4699-9e9c-857095a2664d None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:42.340660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2e970-d6c8-4699-9e9c-857095a2664d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:42.340970 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2e970-d6c8-4699-9e9c-857095a2664d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:42.342398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2e970-d6c8-4699-9e9c-857095a2664d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:42.343243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 533/1064] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:42.359210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 532/1065] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:01:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:42.363736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f60604-298f-4026-9a01-cfcc8c7776e1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:42.366894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2e970-d6c8-4699-9e9c-857095a2664d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:42.431914 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:42.473039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f60604-298f-4026-9a01-cfcc8c7776e1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:42.475066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2e970-d6c8-4699-9e9c-857095a2664d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:42.506948 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f4b6d-9019-48d9-9172-32ca6d444ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:42.507872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f4b6d-9019-48d9-9172-32ca6d444ea4 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:42.508152 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f4b6d-9019-48d9-9172-32ca6d444ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:42.508356 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9774d9-77cf-431e-880d-d4d7b6457786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:42.508492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f4b6d-9019-48d9-9172-32ca6d444ea4 None admin] PATH_INFO: `/v3/projects/06da054e8b7545dfa56780ab56585be8/users/51de59caa3144a1cbf12b20b15e2134f/roles/f70de858079b47cf92a3995fda64bbc0` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:42.509145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9774d9-77cf-431e-880d-d4d7b6457786 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:42.509423 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9774d9-77cf-431e-880d-d4d7b6457786 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:42.509694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9774d9-77cf-431e-880d-d4d7b6457786 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:42.510619 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f4b6d-9019-48d9-9172-32ca6d444ea4 None admin] RBAC: Authorizing `identity:create_grant(project_id=06da054e8b7545dfa56780ab56585be8, user_id=51de59caa3144a1cbf12b20b15e2134f, role_id=f70de858079b47cf92a3995fda64bbc0)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:42.512063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f4b6d-9019-48d9-9172-32ca6d444ea4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:42.520308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9774d9-77cf-431e-880d-d4d7b6457786 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ca9774d9-77cf-431e-880d-d4d7b6457786 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:01:42.527932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:42.535478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:42.536544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:42.536544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:42.536544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:42.536544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:42.536544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:01:42.536544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:01:42.536544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:01:42.536544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:01:42.536544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:01:42.536544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:01:42.536544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:01:42.536544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:42.536544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 533/1066] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:01:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:42.536544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee1bb0-5451-4217-9abc-372d993435a3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:42.537505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee1bb0-5451-4217-9abc-372d993435a3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:42.537505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee1bb0-5451-4217-9abc-372d993435a3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:42.537505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 534/1067] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:42.537779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee1bb0-5451-4217-9abc-372d993435a3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:42.556892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:42.591496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 534/1068] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:01:42 2022] PUT /identity/v3/projects/06da054e8b7545dfa56780ab56585be8/users/51de59caa3144a1cbf12b20b15e2134f/roles/f70de858079b47cf92a3995fda64bbc0 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:01:42.592997 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a096268-5c8c-481c-a16b-c5a1255984b0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:42.593291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a096268-5c8c-481c-a16b-c5a1255984b0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:42.593571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a096268-5c8c-481c-a16b-c5a1255984b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:42.612460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1a096268-5c8c-481c-a16b-c5a1255984b0 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:42.613836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1a096268-5c8c-481c-a16b-c5a1255984b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:42.652761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee1bb0-5451-4217-9abc-372d993435a3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:42.694227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 535/1069] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:42.697304 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafccc9c-be85-48c5-951e-1d6aad509a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:42.697990 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafccc9c-be85-48c5-951e-1d6aad509a6f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:42.698271 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafccc9c-be85-48c5-951e-1d6aad509a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:42.698542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafccc9c-be85-48c5-951e-1d6aad509a6f None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:42.699334 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafccc9c-be85-48c5-951e-1d6aad509a6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:42.699640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafccc9c-be85-48c5-951e-1d6aad509a6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:42.701015 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafccc9c-be85-48c5-951e-1d6aad509a6f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:42.715712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 535/1070] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:01:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:42.716848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a096268-5c8c-481c-a16b-c5a1255984b0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:42.717240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafccc9c-be85-48c5-951e-1d6aad509a6f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:42.782149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:42.803950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:42.827375 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafccc9c-be85-48c5-951e-1d6aad509a6f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:42.829844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a096268-5c8c-481c-a16b-c5a1255984b0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:42.866076 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dadbe4f-8c6a-43ab-85fd-9fd0d0f98ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:42.866856 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadbe4f-8c6a-43ab-85fd-9fd0d0f98ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:42.867140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadbe4f-8c6a-43ab-85fd-9fd0d0f98ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:42.867468 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadbe4f-8c6a-43ab-85fd-9fd0d0f98ca8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:42.868385 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67fa77-73f7-4d00-af48-dc3ed6e18dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:42.869255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67fa77-73f7-4d00-af48-dc3ed6e18dff None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:42.869539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67fa77-73f7-4d00-af48-dc3ed6e18dff None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:42.869868 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67fa77-73f7-4d00-af48-dc3ed6e18dff None admin] PATH_INFO: `/v3/users/ab036aea0f4c4277938c2f0dce2a318d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:42.888359 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67fa77-73f7-4d00-af48-dc3ed6e18dff None admin] RBAC: Authorizing `identity:get_user(user_id=ab036aea0f4c4277938c2f0dce2a318d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:42.889795 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67fa77-73f7-4d00-af48-dc3ed6e18dff None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:42.893204 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 536/1071] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:01:42 2022] GET /identity/v3/users/ab036aea0f4c4277938c2f0dce2a318d => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:42.897813 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67fa77-73f7-4d00-af48-dc3ed6e18dff None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:42.946747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dadbe4f-8c6a-43ab-85fd-9fd0d0f98ca8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:42.948341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dadbe4f-8c6a-43ab-85fd-9fd0d0f98ca8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:43.005097 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67fa77-73f7-4d00-af48-dc3ed6e18dff None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:43.015899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:43.038666 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:43.040272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376ca03-2eec-4501-8638-13220f835dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:43.040933 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376ca03-2eec-4501-8638-13220f835dcc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:43.041212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376ca03-2eec-4501-8638-13220f835dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:43.041488 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376ca03-2eec-4501-8638-13220f835dcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:43.062693 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 536/1072] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:01:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 15:01:43.068631 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dadbe4f-8c6a-43ab-85fd-9fd0d0f98ca8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:43.122973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376ca03-2eec-4501-8638-13220f835dcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:43.124471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376ca03-2eec-4501-8638-13220f835dcc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:43.183797 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dadbe4f-8c6a-43ab-85fd-9fd0d0f98ca8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:43.219272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5e034-4291-482b-9b2c-b25a8954e9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:43.220009 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5e034-4291-482b-9b2c-b25a8954e9ab None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:43.220289 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5e034-4291-482b-9b2c-b25a8954e9ab None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:43.220562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5e034-4291-482b-9b2c-b25a8954e9ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:43.227790 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 537/1073] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:01:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:01:43.231896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5e034-4291-482b-9b2c-b25a8954e9ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:43.233377 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376ca03-2eec-4501-8638-13220f835dcc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:43.233510 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5e034-4291-482b-9b2c-b25a8954e9ab None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:43.245540 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:43.253143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-70b5e034-4291-482b-9b2c-b25a8954e9ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:43.253143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:43.253143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:43.253143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:43.253143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:43.253143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:43.253143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:43.253143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:43.253143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:43.253143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:43.253143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:43.253143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:43.253143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:43.253143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:43.253143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:43.254048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:43.255146 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:43.255146 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:43.255146 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:43.255146 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:43.255146 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 537/1074] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:43.260208 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5e034-4291-482b-9b2c-b25a8954e9ab None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:43.292720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:43.343915 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376ca03-2eec-4501-8638-13220f835dcc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:43.365681 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5e034-4291-482b-9b2c-b25a8954e9ab None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:43.381186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4db746-e47d-4594-b16f-c39520bc2bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:43.382057 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4db746-e47d-4594-b16f-c39520bc2bdb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:43.382342 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4db746-e47d-4594-b16f-c39520bc2bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:43.382621 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4db746-e47d-4594-b16f-c39520bc2bdb None admin] PATH_INFO: `/v3/projects/c71af9998ca146d1ab8c426706dc5392` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:43.383817 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4db746-e47d-4594-b16f-c39520bc2bdb None admin] RBAC: Authorizing `identity:get_project(project_id=c71af9998ca146d1ab8c426706dc5392)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:43.385251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4db746-e47d-4594-b16f-c39520bc2bdb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:43.387026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 538/1075] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:01:43 2022] GET /identity/v3/projects/c71af9998ca146d1ab8c426706dc5392 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:43.393458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4db746-e47d-4594-b16f-c39520bc2bdb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:43.402186 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc776d8a-6cd6-4aba-aa10-9f2e6e4d3715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:43.402861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc776d8a-6cd6-4aba-aa10-9f2e6e4d3715 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:43.403133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc776d8a-6cd6-4aba-aa10-9f2e6e4d3715 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:43.403434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc776d8a-6cd6-4aba-aa10-9f2e6e4d3715 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:43.404155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc776d8a-6cd6-4aba-aa10-9f2e6e4d3715 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:43.404463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc776d8a-6cd6-4aba-aa10-9f2e6e4d3715 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:43.405812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc776d8a-6cd6-4aba-aa10-9f2e6e4d3715 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:43.427208 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 538/1076] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:43.482562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:43.505936 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4db746-e47d-4594-b16f-c39520bc2bdb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:43.542847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c88d07-fda8-4496-a830-8b9490e23de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:43.543542 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c88d07-fda8-4496-a830-8b9490e23de6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:43.543826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c88d07-fda8-4496-a830-8b9490e23de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:43.544098 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c88d07-fda8-4496-a830-8b9490e23de6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:43.544946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c88d07-fda8-4496-a830-8b9490e23de6 None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=c71af9998ca146d1ab8c426706dc5392, user.id=ab036aea0f4c4277938c2f0dce2a318d) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:43.545259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c88d07-fda8-4496-a830-8b9490e23de6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:43.546736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c88d07-fda8-4496-a830-8b9490e23de6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:43.554176 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 539/1077] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:01:43 2022] GET /identity/v3/role_assignments?user.id=ab036aea0f4c4277938c2f0dce2a318d&scope.project.id=c71af9998ca146d1ab8c426706dc5392&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:44.640307 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4d278-a3f5-45cc-a01f-30529b9e3990 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:44.640866 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4d278-a3f5-45cc-a01f-30529b9e3990 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:44.640866 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4d278-a3f5-45cc-a01f-30529b9e3990 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:44.642416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 539/1078] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:44.649668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36220da-9447-42dc-bb52-fbaa4e437178 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:44.650181 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36220da-9447-42dc-bb52-fbaa4e437178 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:44.650276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36220da-9447-42dc-bb52-fbaa4e437178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:44.667059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f36220da-9447-42dc-bb52-fbaa4e437178 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:44.668241 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f36220da-9447-42dc-bb52-fbaa4e437178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:44.732778 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 540/1079] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:44.756893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4d278-a3f5-45cc-a01f-30529b9e3990 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:44.847715 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:44.857730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4d278-a3f5-45cc-a01f-30529b9e3990 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:44.890528 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44bcb0-fbb8-4016-925f-fce57a8bbffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:44.891317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44bcb0-fbb8-4016-925f-fce57a8bbffc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:44.891604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44bcb0-fbb8-4016-925f-fce57a8bbffc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:44.891881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44bcb0-fbb8-4016-925f-fce57a8bbffc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:44.902334 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44bcb0-fbb8-4016-925f-fce57a8bbffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-3e44bcb0-fbb8-4016-925f-fce57a8bbffc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:01:44.910591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:44.911775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:44.912788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:44.912788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:44.912788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:44.912788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:44.912788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:01:44.912788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:01:44.912788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:01:44.912788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:01:44.912788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:01:44.912788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:01:44.912788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 15:01:44.912788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:44.912788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 540/1080] 10.222.0.67 () {56 vars in 1120 bytes} [Fri Jun 24 15:01:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:44.917718 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36220da-9447-42dc-bb52-fbaa4e437178 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:44.985056 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:45.013928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36220da-9447-42dc-bb52-fbaa4e437178 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:45.047627 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ac4ca-2510-4a5d-a5eb-def1cc449bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:45.048382 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ac4ca-2510-4a5d-a5eb-def1cc449bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:45.048668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ac4ca-2510-4a5d-a5eb-def1cc449bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:45.048944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ac4ca-2510-4a5d-a5eb-def1cc449bc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:45.049678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ac4ca-2510-4a5d-a5eb-def1cc449bc0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:45.049984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ac4ca-2510-4a5d-a5eb-def1cc449bc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:45.051617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ac4ca-2510-4a5d-a5eb-def1cc449bc0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:45.059352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 541/1081] 10.222.0.67 () {56 vars in 1115 bytes} [Fri Jun 24 15:01:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:45.065723 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44bcb0-fbb8-4016-925f-fce57a8bbffc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:45.167816 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44bcb0-fbb8-4016-925f-fce57a8bbffc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:45.200362 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455160e7-b8d4-4003-9b68-cc8a64074233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:45.201149 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455160e7-b8d4-4003-9b68-cc8a64074233 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:45.201429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455160e7-b8d4-4003-9b68-cc8a64074233 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:45.201699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455160e7-b8d4-4003-9b68-cc8a64074233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:45.212512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455160e7-b8d4-4003-9b68-cc8a64074233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:45.213943 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455160e7-b8d4-4003-9b68-cc8a64074233 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:45.230714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-455160e7-b8d4-4003-9b68-cc8a64074233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:45.230714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:45.230714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:45.230714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:45.230714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:45.230714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:45.230714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:45.230714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:45.230714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:45.230714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:45.230714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:45.230714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:45.230714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:45.230714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:45.230714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:45.231836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:45.232996 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:45.232996 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:45.232996 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:45.232996 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:45.232996 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 541/1082] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:45.238159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ac4ca-2510-4a5d-a5eb-def1cc449bc0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:45.336269 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ac4ca-2510-4a5d-a5eb-def1cc449bc0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:45.369029 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7f08e-51a4-44ae-9c62-706bd1b8d160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:45.369682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7f08e-51a4-44ae-9c62-706bd1b8d160 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:45.369963 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7f08e-51a4-44ae-9c62-706bd1b8d160 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:45.370237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7f08e-51a4-44ae-9c62-706bd1b8d160 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:45.370945 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7f08e-51a4-44ae-9c62-706bd1b8d160 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:45.371278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7f08e-51a4-44ae-9c62-706bd1b8d160 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:45.372620 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7f08e-51a4-44ae-9c62-706bd1b8d160 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:45.374021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:45.385135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 542/1083] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:45.390733 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455160e7-b8d4-4003-9b68-cc8a64074233 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:45.393652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:45.488997 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455160e7-b8d4-4003-9b68-cc8a64074233 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:45.521053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a28ad-4434-4cb4-8db4-ce569b774b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:45.521843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a28ad-4434-4cb4-8db4-ce569b774b88 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:45.522125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a28ad-4434-4cb4-8db4-ce569b774b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:45.522399 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a28ad-4434-4cb4-8db4-ce569b774b88 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:45.530315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a28ad-4434-4cb4-8db4-ce569b774b88 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:45.531772 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a28ad-4434-4cb4-8db4-ce569b774b88 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-189a28ad-4434-4cb4-8db4-ce569b774b88 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:01:45.545592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:45.546782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:45.547800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:45.547800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:45.547800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:01:45.547800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:01:45.547800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:01:45.547800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:01:45.547800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:01:45.547800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:01:45.547800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 15:01:45.547800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:45.547800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 542/1084] 10.222.0.67 () {56 vars in 1093 bytes} [Fri Jun 24 15:01:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:45.553908 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7f08e-51a4-44ae-9c62-706bd1b8d160 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:45.578704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf4881-817f-4cfb-b9f1-4778995bbd62 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:45.578993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf4881-817f-4cfb-b9f1-4778995bbd62 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:45.579307 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf4881-817f-4cfb-b9f1-4778995bbd62 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:45.581216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 543/1085] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:01:45.586140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98ff1d-9f35-4489-95ec-1712adca0cc9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:45.586429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98ff1d-9f35-4489-95ec-1712adca0cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:45.586707 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98ff1d-9f35-4489-95ec-1712adca0cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:45.603403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ad98ff1d-9f35-4489-95ec-1712adca0cc9 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:45.604593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ad98ff1d-9f35-4489-95ec-1712adca0cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:45.653910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7f08e-51a4-44ae-9c62-706bd1b8d160 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:45.660648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:45.670523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 544/1086] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:01:45.688210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c781450-e01a-4be9-9923-d2b181291edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:45.688882 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c781450-e01a-4be9-9923-d2b181291edb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:45.689166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c781450-e01a-4be9-9923-d2b181291edb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:45.689963 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c781450-e01a-4be9-9923-d2b181291edb None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:45.690784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c781450-e01a-4be9-9923-d2b181291edb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:45.691095 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c781450-e01a-4be9-9923-d2b181291edb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:45.692704 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c781450-e01a-4be9-9923-d2b181291edb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:45.693520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98ff1d-9f35-4489-95ec-1712adca0cc9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:45.710733 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 543/1087] 10.222.0.67 () {56 vars in 1133 bytes} [Fri Jun 24 15:01:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:45.716296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c781450-e01a-4be9-9923-d2b181291edb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:45.767603 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:45.795767 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98ff1d-9f35-4489-95ec-1712adca0cc9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:45.813738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c781450-e01a-4be9-9923-d2b181291edb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:45.832382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34e469-d0da-4b4c-89d7-acdeb3f614bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:45.833367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34e469-d0da-4b4c-89d7-acdeb3f614bb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:45.833783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34e469-d0da-4b4c-89d7-acdeb3f614bb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:45.834188 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34e469-d0da-4b4c-89d7-acdeb3f614bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:45.845582 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34e469-d0da-4b4c-89d7-acdeb3f614bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:01:45.849621 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d963855-1fcf-41de-a09f-5c62b485f188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:45.850264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d963855-1fcf-41de-a09f-5c62b485f188 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:45.850547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d963855-1fcf-41de-a09f-5c62b485f188 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:45.850822 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d963855-1fcf-41de-a09f-5c62b485f188 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-fd34e469-d0da-4b4c-89d7-acdeb3f614bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:01:45.854467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:45.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:45.856726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:45.856726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:45.856726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:45.856726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:45.856726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:01:45.856726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:01:45.856726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:01:45.856726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:01:45.856726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:01:45.856726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:01:45.856726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:01:45.856726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:01:45.859483 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 545/1088] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:01:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:45.861632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34e469-d0da-4b4c-89d7-acdeb3f614bb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:45.913021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:45.916656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d963855-1fcf-41de-a09f-5c62b485f188 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:45.918095 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d963855-1fcf-41de-a09f-5c62b485f188 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:45.959817 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34e469-d0da-4b4c-89d7-acdeb3f614bb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:45.968132 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:45.992638 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975aa1c7-7c21-41b2-8676-ee178306013d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:45.993285 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975aa1c7-7c21-41b2-8676-ee178306013d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:45.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975aa1c7-7c21-41b2-8676-ee178306013d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:45.993843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975aa1c7-7c21-41b2-8676-ee178306013d None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:45.994590 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975aa1c7-7c21-41b2-8676-ee178306013d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:45.994898 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975aa1c7-7c21-41b2-8676-ee178306013d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:45.996287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975aa1c7-7c21-41b2-8676-ee178306013d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:46.005099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 546/1089] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:01:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:46.010750 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975aa1c7-7c21-41b2-8676-ee178306013d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:46.018210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 544/1090] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:01:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 15:01:46.023675 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d963855-1fcf-41de-a09f-5c62b485f188 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:46.110639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975aa1c7-7c21-41b2-8676-ee178306013d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:46.122376 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d963855-1fcf-41de-a09f-5c62b485f188 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:46.124891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:46.145658 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c74e4-8f2c-4754-8e7f-92e50a4fe40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:46.146522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c74e4-8f2c-4754-8e7f-92e50a4fe40b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:46.146802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c74e4-8f2c-4754-8e7f-92e50a4fe40b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:46.147073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c74e4-8f2c-4754-8e7f-92e50a4fe40b None admin] PATH_INFO: `/v3/users/ab036aea0f4c4277938c2f0dce2a318d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:46.148311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c74e4-8f2c-4754-8e7f-92e50a4fe40b None admin] RBAC: Authorizing `identity:get_user(user_id=ab036aea0f4c4277938c2f0dce2a318d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:46.149794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c74e4-8f2c-4754-8e7f-92e50a4fe40b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:46.155484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 547/1091] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:01:46 2022] GET /identity/v3/users/ab036aea0f4c4277938c2f0dce2a318d => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:46.155839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c241e5-2e00-4995-85ab-59cf62e54f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:46.156749 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c241e5-2e00-4995-85ab-59cf62e54f3a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:46.157016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c241e5-2e00-4995-85ab-59cf62e54f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:46.157218 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c74e4-8f2c-4754-8e7f-92e50a4fe40b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:46.157314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c241e5-2e00-4995-85ab-59cf62e54f3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:46.167848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c241e5-2e00-4995-85ab-59cf62e54f3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:46.169292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c241e5-2e00-4995-85ab-59cf62e54f3a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:46.188274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-20c241e5-2e00-4995-85ab-59cf62e54f3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:46.188274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:46.188274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:46.188274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:46.188274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:46.188274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:46.188274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:46.188274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:46.188274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:46.188274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:46.188274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:46.188274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:46.188274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:01:46.188274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:01:46.188274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:46.189254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:46.191013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:01:46.191013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:01:46.191013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:01:46.191013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:46.191013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 545/1092] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:01:46.196132 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c241e5-2e00-4995-85ab-59cf62e54f3a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:46.199849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:46.255680 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c74e4-8f2c-4754-8e7f-92e50a4fe40b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:46.289235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6d7b2-7226-46e0-9999-d409392accde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:46.290027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6d7b2-7226-46e0-9999-d409392accde None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:46.290313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6d7b2-7226-46e0-9999-d409392accde None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:46.290588 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6d7b2-7226-46e0-9999-d409392accde None admin] PATH_INFO: `/v3/projects/c71af9998ca146d1ab8c426706dc5392` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:46.291785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6d7b2-7226-46e0-9999-d409392accde None admin] RBAC: Authorizing `identity:get_project(project_id=c71af9998ca146d1ab8c426706dc5392)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:46.293206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6d7b2-7226-46e0-9999-d409392accde None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:46.294250 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c241e5-2e00-4995-85ab-59cf62e54f3a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:46.295059 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 548/1093] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:01:46 2022] GET /identity/v3/projects/c71af9998ca146d1ab8c426706dc5392 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:46.300801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6d7b2-7226-46e0-9999-d409392accde None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:46.327983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d02080-a6cd-4ad8-93e8-881d3428585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:46.328642 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d02080-a6cd-4ad8-93e8-881d3428585a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:46.328927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d02080-a6cd-4ad8-93e8-881d3428585a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:46.329204 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d02080-a6cd-4ad8-93e8-881d3428585a None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:46.330493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d02080-a6cd-4ad8-93e8-881d3428585a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:46.330932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d02080-a6cd-4ad8-93e8-881d3428585a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:46.332344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d02080-a6cd-4ad8-93e8-881d3428585a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:46.348538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 546/1094] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:46.349521 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d02080-a6cd-4ad8-93e8-881d3428585a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:46.353346 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:46.403591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6d7b2-7226-46e0-9999-d409392accde None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:46.409671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:46.436632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd9f9f-5c99-49c6-8ee9-86bcd16877c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:46.437522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd9f9f-5c99-49c6-8ee9-86bcd16877c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:46.437805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd9f9f-5c99-49c6-8ee9-86bcd16877c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:46.438083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd9f9f-5c99-49c6-8ee9-86bcd16877c1 None admin] PATH_INFO: `/v3/projects/c71af9998ca146d1ab8c426706dc5392/users/ab036aea0f4c4277938c2f0dce2a318d/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:46.440959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd9f9f-5c99-49c6-8ee9-86bcd16877c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c71af9998ca146d1ab8c426706dc5392, user_id=ab036aea0f4c4277938c2f0dce2a318d, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:46.442409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd9f9f-5c99-49c6-8ee9-86bcd16877c1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:46.447339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d02080-a6cd-4ad8-93e8-881d3428585a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:46.480468 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cae112-3a6c-463d-b64f-cb525f07fba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:46.481262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cae112-3a6c-463d-b64f-cb525f07fba1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:46.481547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cae112-3a6c-463d-b64f-cb525f07fba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:46.481819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cae112-3a6c-463d-b64f-cb525f07fba1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:46.488438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cae112-3a6c-463d-b64f-cb525f07fba1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:46.489862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cae112-3a6c-463d-b64f-cb525f07fba1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:46.504343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-64cae112-3a6c-463d-b64f-cb525f07fba1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:46.504343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:46.504343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:46.504343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:46.504343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:46.504343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:46.504343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:46.504343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:46.504343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:46.504343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:46.504343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:46.504343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:46.504343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:01:46.504343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:01:46.504343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:46.505407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:46.506373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:01:46.506373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:01:46.506373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:01:46.506373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:01:46.506373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:01:46.506373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:01:46.506373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:01:46.506373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:46.506373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 547/1095] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:01:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:46.510584 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 549/1096] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:01:46 2022] PUT /identity/v3/projects/c71af9998ca146d1ab8c426706dc5392/users/ab036aea0f4c4277938c2f0dce2a318d/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:01:46.511201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cae112-3a6c-463d-b64f-cb525f07fba1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:46.621426 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cae112-3a6c-463d-b64f-cb525f07fba1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:46.648278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:46.662586 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afaec5-c5ed-44d3-b8f0-d4865ee1ea4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:46.665151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afaec5-c5ed-44d3-b8f0-d4865ee1ea4b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:46.665151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afaec5-c5ed-44d3-b8f0-d4865ee1ea4b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:46.665151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afaec5-c5ed-44d3-b8f0-d4865ee1ea4b None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:46.665151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afaec5-c5ed-44d3-b8f0-d4865ee1ea4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:46.665151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afaec5-c5ed-44d3-b8f0-d4865ee1ea4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:46.666421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afaec5-c5ed-44d3-b8f0-d4865ee1ea4b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:46.678873 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 548/1097] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:01:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:01:46.685348 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd9f9f-5c99-49c6-8ee9-86bcd16877c1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:46.788852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd9f9f-5c99-49c6-8ee9-86bcd16877c1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:46.799238 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:46.823479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd13c5-a77e-453b-9c22-633a375a1c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:46.824128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd13c5-a77e-453b-9c22-633a375a1c3b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:46.824420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd13c5-a77e-453b-9c22-633a375a1c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:46.824894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd13c5-a77e-453b-9c22-633a375a1c3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:46.825882 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd13c5-a77e-453b-9c22-633a375a1c3b None admin] RBAC: Adding query filter params (role.id=f70de858079b47cf92a3995fda64bbc0, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=51de59caa3144a1cbf12b20b15e2134f) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:46.826615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd13c5-a77e-453b-9c22-633a375a1c3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:46.828057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd13c5-a77e-453b-9c22-633a375a1c3b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:46.834092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 550/1098] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:01:46 2022] GET /identity/v3/role_assignments?user.id=51de59caa3144a1cbf12b20b15e2134f&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=f70de858079b47cf92a3995fda64bbc0 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:48.535614 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c2edc-e4d8-4f8c-a4e9-5ca7cf7028c1 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:48.536166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c2edc-e4d8-4f8c-a4e9-5ca7cf7028c1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:48.536166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c2edc-e4d8-4f8c-a4e9-5ca7cf7028c1 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:48.537549 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 549/1099] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:48.542701 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa531c-1f1c-415a-ba55-912b35cca8d4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:48.543068 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa531c-1f1c-415a-ba55-912b35cca8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:48.543303 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa531c-1f1c-415a-ba55-912b35cca8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:48.560374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5aaa531c-1f1c-415a-ba55-912b35cca8d4 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:48.561535 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5aaa531c-1f1c-415a-ba55-912b35cca8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:48.625728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 551/1100] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:48.650143 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c2edc-e4d8-4f8c-a4e9-5ca7cf7028c1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:48.748272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c2edc-e4d8-4f8c-a4e9-5ca7cf7028c1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:48.781125 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98700116-ad45-4e62-b9bd-008d61c31b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:48.781949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98700116-ad45-4e62-b9bd-008d61c31b2a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:48.782224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98700116-ad45-4e62-b9bd-008d61c31b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:48.782493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98700116-ad45-4e62-b9bd-008d61c31b2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:48.792862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98700116-ad45-4e62-b9bd-008d61c31b2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-98700116-ad45-4e62-b9bd-008d61c31b2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:01:48.800744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:01:48.802108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:01:48.803159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:01:48.803159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:01:48.803159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:01:48.803159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:01:48.803159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:01:48.803159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:01:48.803159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:01:48.803159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:01:48.803159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:01:48.803159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:01:48.803159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:01:48.803159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:01:48.803159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 550/1101] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:01:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:01:48.808332 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa531c-1f1c-415a-ba55-912b35cca8d4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:48.905522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa531c-1f1c-415a-ba55-912b35cca8d4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:48.907778 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:48.938368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385f52a-fbdf-450a-8a9f-88c256080c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:48.939040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385f52a-fbdf-450a-8a9f-88c256080c38 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:48.940368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385f52a-fbdf-450a-8a9f-88c256080c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:48.940758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385f52a-fbdf-450a-8a9f-88c256080c38 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:48.941730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385f52a-fbdf-450a-8a9f-88c256080c38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:48.942099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385f52a-fbdf-450a-8a9f-88c256080c38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:48.943792 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385f52a-fbdf-450a-8a9f-88c256080c38 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:48.951899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 552/1102] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:01:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:48.958248 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98700116-ad45-4e62-b9bd-008d61c31b2a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:49.003938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:49.056986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98700116-ad45-4e62-b9bd-008d61c31b2a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:49.090251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025eb24d-242b-4a3f-bc7a-9ecef5c26672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:49.091124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025eb24d-242b-4a3f-bc7a-9ecef5c26672 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:49.091436 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025eb24d-242b-4a3f-bc7a-9ecef5c26672 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:49.091711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025eb24d-242b-4a3f-bc7a-9ecef5c26672 None admin] PATH_INFO: `/v3/users/ab036aea0f4c4277938c2f0dce2a318d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:49.092905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025eb24d-242b-4a3f-bc7a-9ecef5c26672 None admin] RBAC: Authorizing `identity:get_user(user_id=ab036aea0f4c4277938c2f0dce2a318d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:49.094335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025eb24d-242b-4a3f-bc7a-9ecef5c26672 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:49.096347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 551/1103] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:01:48 2022] GET /identity/v3/users/ab036aea0f4c4277938c2f0dce2a318d => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:49.101920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385f52a-fbdf-450a-8a9f-88c256080c38 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:49.199073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385f52a-fbdf-450a-8a9f-88c256080c38 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:49.230649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328b65e-bbff-463c-926f-642dbb266fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:49.231304 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328b65e-bbff-463c-926f-642dbb266fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:49.231591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328b65e-bbff-463c-926f-642dbb266fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:49.231867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328b65e-bbff-463c-926f-642dbb266fd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:49.264147 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:49.296065 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328b65e-bbff-463c-926f-642dbb266fd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:49.297492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328b65e-bbff-463c-926f-642dbb266fd3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:49.391046 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 553/1104] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:01:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:01:49.396806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025eb24d-242b-4a3f-bc7a-9ecef5c26672 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:49.491690 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025eb24d-242b-4a3f-bc7a-9ecef5c26672 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:49.504395 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:49.524379 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5951b-f26d-4ddd-9140-a8f2a283f37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:49.525225 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5951b-f26d-4ddd-9140-a8f2a283f37c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:49.525502 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5951b-f26d-4ddd-9140-a8f2a283f37c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:49.525772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5951b-f26d-4ddd-9140-a8f2a283f37c None admin] PATH_INFO: `/v3/projects/c71af9998ca146d1ab8c426706dc5392` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:49.526917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5951b-f26d-4ddd-9140-a8f2a283f37c None admin] RBAC: Authorizing `identity:get_project(project_id=c71af9998ca146d1ab8c426706dc5392)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:49.528359 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5951b-f26d-4ddd-9140-a8f2a283f37c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:49.530009 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 552/1105] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:01:49 2022] GET /identity/v3/projects/c71af9998ca146d1ab8c426706dc5392 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:49.536418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328b65e-bbff-463c-926f-642dbb266fd3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:49.599909 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:49.633044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328b65e-bbff-463c-926f-642dbb266fd3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:49.664746 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2a175-9019-4ffc-be01-3a2d19131e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:49.665325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2a175-9019-4ffc-be01-3a2d19131e36 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:49.665610 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2a175-9019-4ffc-be01-3a2d19131e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:49.665885 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2a175-9019-4ffc-be01-3a2d19131e36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:49.666699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2a175-9019-4ffc-be01-3a2d19131e36 None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=c71af9998ca146d1ab8c426706dc5392, user.id=ab036aea0f4c4277938c2f0dce2a318d) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:49.667008 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2a175-9019-4ffc-be01-3a2d19131e36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:49.668443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2a175-9019-4ffc-be01-3a2d19131e36 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:49.674743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 554/1106] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:01:49 2022] GET /identity/v3/role_assignments?user.id=ab036aea0f4c4277938c2f0dce2a318d&scope.project.id=c71af9998ca146d1ab8c426706dc5392&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:51.749002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0f4ec-9eff-4b17-a969-8fc16d884cdb None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:51.752220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0f4ec-9eff-4b17-a969-8fc16d884cdb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:51.752220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0f4ec-9eff-4b17-a969-8fc16d884cdb None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:51.752220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 553/1107] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:51.756576 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff0502-3689-49d9-90c6-8d5ade9e7451 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:51.757002 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff0502-3689-49d9-90c6-8d5ade9e7451 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:51.757115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff0502-3689-49d9-90c6-8d5ade9e7451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:51.774116 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-98ff0502-3689-49d9-90c6-8d5ade9e7451 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:51.775317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-98ff0502-3689-49d9-90c6-8d5ade9e7451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:51.839919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 555/1108] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:51.864377 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0f4ec-9eff-4b17-a969-8fc16d884cdb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:51.963160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0f4ec-9eff-4b17-a969-8fc16d884cdb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:51.996336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476b499-9d43-4ebf-9d9d-cd41e33e4bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:51.997110 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476b499-9d43-4ebf-9d9d-cd41e33e4bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:51.997394 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476b499-9d43-4ebf-9d9d-cd41e33e4bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:51.997672 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476b499-9d43-4ebf-9d9d-cd41e33e4bb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:51.998838 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476b499-9d43-4ebf-9d9d-cd41e33e4bb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:52.000345 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476b499-9d43-4ebf-9d9d-cd41e33e4bb9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:52.002007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 554/1109] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:01:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:52.007737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff0502-3689-49d9-90c6-8d5ade9e7451 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:52.063599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:52.104311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff0502-3689-49d9-90c6-8d5ade9e7451 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:52.136776 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8bea5-24c9-4dd7-880c-dfdcb0aeeed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:52.137443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8bea5-24c9-4dd7-880c-dfdcb0aeeed4 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:52.137728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8bea5-24c9-4dd7-880c-dfdcb0aeeed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:52.138007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8bea5-24c9-4dd7-880c-dfdcb0aeeed4 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:52.138715 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc8bea5-24c9-4dd7-880c-dfdcb0aeeed4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:52.140171 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc8bea5-24c9-4dd7-880c-dfdcb0aeeed4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:52.230570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 556/1110] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 15:01:52 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:01:54.334749 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a65754-3191-4b78-b1f9-c2e611c07ded None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:54.335357 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a65754-3191-4b78-b1f9-c2e611c07ded None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:54.335357 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a65754-3191-4b78-b1f9-c2e611c07ded None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:54.336709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 555/1111] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:54.341946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f5d3c-19dd-45cc-8e9a-326b7a521325 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:54.342397 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f5d3c-19dd-45cc-8e9a-326b7a521325 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:54.342501 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f5d3c-19dd-45cc-8e9a-326b7a521325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:54.359016 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-568f5d3c-19dd-45cc-8e9a-326b7a521325 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:54.360142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-568f5d3c-19dd-45cc-8e9a-326b7a521325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:54.428046 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 557/1112] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:54.453141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a65754-3191-4b78-b1f9-c2e611c07ded None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:54.502764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:54.553764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a65754-3191-4b78-b1f9-c2e611c07ded None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:54.587093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42674249-b965-4afd-9678-43270961c1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:54.587965 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42674249-b965-4afd-9678-43270961c1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:54.588252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42674249-b965-4afd-9678-43270961c1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:54.588551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42674249-b965-4afd-9678-43270961c1f1 None admin] PATH_INFO: `/v3/roles/c077ebdf7f214d7585dfc1570eecf882` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:54.590026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42674249-b965-4afd-9678-43270961c1f1 None admin] RBAC: Authorizing `identity:get_role(role_id=c077ebdf7f214d7585dfc1570eecf882)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:54.591428 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42674249-b965-4afd-9678-43270961c1f1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:54.592797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 556/1113] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:01:54 2022] GET /identity/v3/roles/c077ebdf7f214d7585dfc1570eecf882 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:54.599450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f5d3c-19dd-45cc-8e9a-326b7a521325 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:54.643864 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:54.695121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f5d3c-19dd-45cc-8e9a-326b7a521325 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:54.728421 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c9962-4545-4476-8c8f-fca30cbad56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:54.729289 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c9962-4545-4476-8c8f-fca30cbad56e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:54.729570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c9962-4545-4476-8c8f-fca30cbad56e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:54.729843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c9962-4545-4476-8c8f-fca30cbad56e None admin] PATH_INFO: `/v3/users/20849fcc587942de8fd5498b3dc79f81` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:54.743022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c9962-4545-4476-8c8f-fca30cbad56e None admin] RBAC: Authorizing `identity:get_user(user_id=20849fcc587942de8fd5498b3dc79f81)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:54.744527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c9962-4545-4476-8c8f-fca30cbad56e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:54.746144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 558/1114] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:01:54 2022] GET /identity/v3/users/20849fcc587942de8fd5498b3dc79f81 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:54.752273 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42674249-b965-4afd-9678-43270961c1f1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:54.848772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42674249-b965-4afd-9678-43270961c1f1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:54.865072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:54.881792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10801897-4804-4a72-89c5-b12bbb63bcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:54.882428 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10801897-4804-4a72-89c5-b12bbb63bcb2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:54.882709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10801897-4804-4a72-89c5-b12bbb63bcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:54.882985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10801897-4804-4a72-89c5-b12bbb63bcb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:54.948531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10801897-4804-4a72-89c5-b12bbb63bcb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:54.949950 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10801897-4804-4a72-89c5-b12bbb63bcb2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:55.046324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 557/1115] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:01:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:01:55.052853 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c9962-4545-4476-8c8f-fca30cbad56e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:55.149905 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c9962-4545-4476-8c8f-fca30cbad56e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:55.152263 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:55.182596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02987a92-d85c-43c2-b01a-0699797d5c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:55.183490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02987a92-d85c-43c2-b01a-0699797d5c29 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:55.183769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02987a92-d85c-43c2-b01a-0699797d5c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:55.184038 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02987a92-d85c-43c2-b01a-0699797d5c29 None admin] PATH_INFO: `/v3/projects/c71af9998ca146d1ab8c426706dc5392` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:55.185296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02987a92-d85c-43c2-b01a-0699797d5c29 None admin] RBAC: Authorizing `identity:get_project(project_id=c71af9998ca146d1ab8c426706dc5392)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:55.186718 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02987a92-d85c-43c2-b01a-0699797d5c29 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:55.188414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 559/1116] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:01:55 2022] GET /identity/v3/projects/c71af9998ca146d1ab8c426706dc5392 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:55.194702 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10801897-4804-4a72-89c5-b12bbb63bcb2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:55.212898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:55.294150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10801897-4804-4a72-89c5-b12bbb63bcb2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:55.327171 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72071c83-0b65-4838-bb4a-768202802e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:55.327848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72071c83-0b65-4838-bb4a-768202802e11 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:55.328133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72071c83-0b65-4838-bb4a-768202802e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:55.328412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72071c83-0b65-4838-bb4a-768202802e11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:55.329229 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72071c83-0b65-4838-bb4a-768202802e11 None admin] RBAC: Adding query filter params (role.id=c077ebdf7f214d7585dfc1570eecf882, scope.project.id=c71af9998ca146d1ab8c426706dc5392, user.id=20849fcc587942de8fd5498b3dc79f81) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:01:55.330090 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72071c83-0b65-4838-bb4a-768202802e11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:55.331834 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72071c83-0b65-4838-bb4a-768202802e11 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:55.337124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 558/1117] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:01:55 2022] GET /identity/v3/role_assignments?user.id=20849fcc587942de8fd5498b3dc79f81&scope.project.id=c71af9998ca146d1ab8c426706dc5392&role.id=c077ebdf7f214d7585dfc1570eecf882 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:57.361871 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c83901-2905-4fa1-bfe3-23608f7df9c4 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:57.362373 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c83901-2905-4fa1-bfe3-23608f7df9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:57.362476 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c83901-2905-4fa1-bfe3-23608f7df9c4 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:57.363838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 560/1118] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:01:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:01:57.369102 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997d546-61a9-4dc8-8bd2-f5a48c0f2b03 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:57.369484 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997d546-61a9-4dc8-8bd2-f5a48c0f2b03 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:57.369645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997d546-61a9-4dc8-8bd2-f5a48c0f2b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:57.385744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1997d546-61a9-4dc8-8bd2-f5a48c0f2b03 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:01:57.386905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1997d546-61a9-4dc8-8bd2-f5a48c0f2b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:57.452022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 559/1119] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:01:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:01:57.476384 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c83901-2905-4fa1-bfe3-23608f7df9c4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:57.575767 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c83901-2905-4fa1-bfe3-23608f7df9c4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:57.608653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36424d2f-8846-4a8f-9a88-16bd86164bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:57.609499 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36424d2f-8846-4a8f-9a88-16bd86164bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:57.609787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36424d2f-8846-4a8f-9a88-16bd86164bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:57.610066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36424d2f-8846-4a8f-9a88-16bd86164bc9 None admin] PATH_INFO: `/v3/roles/c077ebdf7f214d7585dfc1570eecf882` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:57.612198 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36424d2f-8846-4a8f-9a88-16bd86164bc9 None admin] RBAC: Authorizing `identity:get_role(role_id=c077ebdf7f214d7585dfc1570eecf882)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:57.613506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36424d2f-8846-4a8f-9a88-16bd86164bc9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:57.614839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 561/1120] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:01:57 2022] GET /identity/v3/roles/c077ebdf7f214d7585dfc1570eecf882 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:57.620706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997d546-61a9-4dc8-8bd2-f5a48c0f2b03 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:57.632027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:57.717396 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997d546-61a9-4dc8-8bd2-f5a48c0f2b03 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:57.750385 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf34426-cc28-4d22-94ce-69d27c07444d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:57.751292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf34426-cc28-4d22-94ce-69d27c07444d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:57.751585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf34426-cc28-4d22-94ce-69d27c07444d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:57.751864 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf34426-cc28-4d22-94ce-69d27c07444d None admin] PATH_INFO: `/v3/users/20849fcc587942de8fd5498b3dc79f81` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:57.753024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf34426-cc28-4d22-94ce-69d27c07444d None admin] RBAC: Authorizing `identity:get_user(user_id=20849fcc587942de8fd5498b3dc79f81)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:57.754444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf34426-cc28-4d22-94ce-69d27c07444d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:57.756064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 560/1121] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:01:57 2022] GET /identity/v3/users/20849fcc587942de8fd5498b3dc79f81 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:57.761950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36424d2f-8846-4a8f-9a88-16bd86164bc9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:57.803362 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:57.858343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36424d2f-8846-4a8f-9a88-16bd86164bc9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:57.891084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cbe57-1e93-44d1-803e-8558eb6be157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:57.892066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cbe57-1e93-44d1-803e-8558eb6be157 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:57.892475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cbe57-1e93-44d1-803e-8558eb6be157 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:57.892825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cbe57-1e93-44d1-803e-8558eb6be157 None admin] PATH_INFO: `/v3/projects/c71af9998ca146d1ab8c426706dc5392` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:57.893994 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877cbe57-1e93-44d1-803e-8558eb6be157 None admin] RBAC: Authorizing `identity:get_project(project_id=c71af9998ca146d1ab8c426706dc5392)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:57.895512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877cbe57-1e93-44d1-803e-8558eb6be157 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:57.897224 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 562/1122] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:01:57 2022] GET /identity/v3/projects/c71af9998ca146d1ab8c426706dc5392 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:01:57.903003 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf34426-cc28-4d22-94ce-69d27c07444d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:57.979069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:58.000384 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf34426-cc28-4d22-94ce-69d27c07444d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:58.034241 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67a499-16f8-4359-b93d-a2cb9a6dc718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:58.035113 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67a499-16f8-4359-b93d-a2cb9a6dc718 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:58.035423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67a499-16f8-4359-b93d-a2cb9a6dc718 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:58.035701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67a499-16f8-4359-b93d-a2cb9a6dc718 None admin] PATH_INFO: `/v3/projects/c71af9998ca146d1ab8c426706dc5392/users/20849fcc587942de8fd5498b3dc79f81/roles/c077ebdf7f214d7585dfc1570eecf882` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:58.037863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67a499-16f8-4359-b93d-a2cb9a6dc718 None admin] RBAC: Authorizing `identity:create_grant(project_id=c71af9998ca146d1ab8c426706dc5392, user_id=20849fcc587942de8fd5498b3dc79f81, role_id=c077ebdf7f214d7585dfc1570eecf882)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:58.039318 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67a499-16f8-4359-b93d-a2cb9a6dc718 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:58.145864 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 561/1123] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:01:57 2022] PUT /identity/v3/projects/c71af9998ca146d1ab8c426706dc5392/users/20849fcc587942de8fd5498b3dc79f81/roles/c077ebdf7f214d7585dfc1570eecf882 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:02:00.154327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded1b29-e626-4910-bb2b-f7054e50b040 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:00.154841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded1b29-e626-4910-bb2b-f7054e50b040 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:00.154960 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded1b29-e626-4910-bb2b-f7054e50b040 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:00.156275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 563/1124] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:02:00.161557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f50fc3-b9cb-44c1-916a-de05e37a1556 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:00.161956 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f50fc3-b9cb-44c1-916a-de05e37a1556 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:00.162114 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f50fc3-b9cb-44c1-916a-de05e37a1556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:00.178565 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-70f50fc3-b9cb-44c1-916a-de05e37a1556 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:02:00.179765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-70f50fc3-b9cb-44c1-916a-de05e37a1556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:02:00.245569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 562/1125] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:02:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:02:00.269686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded1b29-e626-4910-bb2b-f7054e50b040 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:00.367407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded1b29-e626-4910-bb2b-f7054e50b040 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:00.376079 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:00.400036 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9112a-be71-4206-ad93-5f1129ebc6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:00.400885 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9112a-be71-4206-ad93-5f1129ebc6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:00.401169 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9112a-be71-4206-ad93-5f1129ebc6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:00.401442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9112a-be71-4206-ad93-5f1129ebc6d3 None admin] PATH_INFO: `/v3/roles/c077ebdf7f214d7585dfc1570eecf882` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:00.402936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9112a-be71-4206-ad93-5f1129ebc6d3 None admin] RBAC: Authorizing `identity:get_role(role_id=c077ebdf7f214d7585dfc1570eecf882)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:00.404335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9112a-be71-4206-ad93-5f1129ebc6d3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:00.405693 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 564/1126] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:02:00 2022] GET /identity/v3/roles/c077ebdf7f214d7585dfc1570eecf882 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:00.411773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f50fc3-b9cb-44c1-916a-de05e37a1556 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:00.474523 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:00.507146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f50fc3-b9cb-44c1-916a-de05e37a1556 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:00.541144 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7b5fd-185d-470d-8f99-6aa57dc6b6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:00.542010 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7b5fd-185d-470d-8f99-6aa57dc6b6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:00.542285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7b5fd-185d-470d-8f99-6aa57dc6b6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:00.542557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7b5fd-185d-470d-8f99-6aa57dc6b6b6 None admin] PATH_INFO: `/v3/users/20849fcc587942de8fd5498b3dc79f81` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:00.543743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7b5fd-185d-470d-8f99-6aa57dc6b6b6 None admin] RBAC: Authorizing `identity:get_user(user_id=20849fcc587942de8fd5498b3dc79f81)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:00.545151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7b5fd-185d-470d-8f99-6aa57dc6b6b6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:00.546759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 563/1127] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:02:00 2022] GET /identity/v3/users/20849fcc587942de8fd5498b3dc79f81 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:00.553033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9112a-be71-4206-ad93-5f1129ebc6d3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:00.649273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9112a-be71-4206-ad93-5f1129ebc6d3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:00.681386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df214005-e09d-4d15-9465-a4ee8f2cd600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:00.682020 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df214005-e09d-4d15-9465-a4ee8f2cd600 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:00.682303 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df214005-e09d-4d15-9465-a4ee8f2cd600 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:00.682579 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df214005-e09d-4d15-9465-a4ee8f2cd600 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:00.725451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:00.746765 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df214005-e09d-4d15-9465-a4ee8f2cd600 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:00.748229 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df214005-e09d-4d15-9465-a4ee8f2cd600 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:00.843903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 565/1128] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:02:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:02:00.849961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7b5fd-185d-470d-8f99-6aa57dc6b6b6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:00.946356 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7b5fd-185d-470d-8f99-6aa57dc6b6b6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:00.978849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89569149-42ee-419d-9531-67ef3e4b3759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:00.979720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89569149-42ee-419d-9531-67ef3e4b3759 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:00.980010 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89569149-42ee-419d-9531-67ef3e4b3759 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:00.980283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89569149-42ee-419d-9531-67ef3e4b3759 None admin] PATH_INFO: `/v3/projects/c71af9998ca146d1ab8c426706dc5392` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:00.981441 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89569149-42ee-419d-9531-67ef3e4b3759 None admin] RBAC: Authorizing `identity:get_project(project_id=c71af9998ca146d1ab8c426706dc5392)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:00.982852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89569149-42ee-419d-9531-67ef3e4b3759 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:00.984538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 564/1129] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:02:00 2022] GET /identity/v3/projects/c71af9998ca146d1ab8c426706dc5392 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:00.991157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df214005-e09d-4d15-9465-a4ee8f2cd600 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:01.065761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:01.087675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df214005-e09d-4d15-9465-a4ee8f2cd600 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:01.121380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fab071-2713-4a1c-b0d4-c80a142582e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:01.122029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fab071-2713-4a1c-b0d4-c80a142582e2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:01.122315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fab071-2713-4a1c-b0d4-c80a142582e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:01.122594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fab071-2713-4a1c-b0d4-c80a142582e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:01.123447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fab071-2713-4a1c-b0d4-c80a142582e2 None admin] RBAC: Adding query filter params (role.id=c077ebdf7f214d7585dfc1570eecf882, scope.project.id=c71af9998ca146d1ab8c426706dc5392, user.id=20849fcc587942de8fd5498b3dc79f81) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:02:01.123763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fab071-2713-4a1c-b0d4-c80a142582e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:01.125186 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fab071-2713-4a1c-b0d4-c80a142582e2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:01.131884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 566/1130] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:02:00 2022] GET /identity/v3/role_assignments?user.id=20849fcc587942de8fd5498b3dc79f81&scope.project.id=c71af9998ca146d1ab8c426706dc5392&role.id=c077ebdf7f214d7585dfc1570eecf882 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:03.166715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b159c-1bb3-4ae5-9630-6b54b48b2e0b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:03.167252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b159c-1bb3-4ae5-9630-6b54b48b2e0b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:03.167450 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b159c-1bb3-4ae5-9630-6b54b48b2e0b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:03.168702 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 565/1131] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:02:03.174440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f9c59-7fc6-44d7-9855-9dc7977ce7a6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:03.174860 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f9c59-7fc6-44d7-9855-9dc7977ce7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:03.174996 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f9c59-7fc6-44d7-9855-9dc7977ce7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:03.191456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ba6f9c59-7fc6-44d7-9855-9dc7977ce7a6 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:02:03.192627 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ba6f9c59-7fc6-44d7-9855-9dc7977ce7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:02:03.256615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 567/1132] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:02:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:02:03.280235 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b159c-1bb3-4ae5-9630-6b54b48b2e0b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:03.284574 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:03.379046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b159c-1bb3-4ae5-9630-6b54b48b2e0b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:03.411912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f3900-7b59-4ea2-8b65-ea10ca9e3e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:03.412687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f3900-7b59-4ea2-8b65-ea10ca9e3e93 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:03.412970 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f3900-7b59-4ea2-8b65-ea10ca9e3e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:03.413247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f3900-7b59-4ea2-8b65-ea10ca9e3e93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:03.414402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f3900-7b59-4ea2-8b65-ea10ca9e3e93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:03.415851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f3900-7b59-4ea2-8b65-ea10ca9e3e93 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:03.417438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 566/1133] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:02:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:03.423158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f9c59-7fc6-44d7-9855-9dc7977ce7a6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:03.488024 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:03.518854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f9c59-7fc6-44d7-9855-9dc7977ce7a6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:03.551781 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a32607-dee7-4122-8a21-fbd2a19c2499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:03.552422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a32607-dee7-4122-8a21-fbd2a19c2499 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:03.552704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a32607-dee7-4122-8a21-fbd2a19c2499 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:03.552977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a32607-dee7-4122-8a21-fbd2a19c2499 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:03.553684 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a32607-dee7-4122-8a21-fbd2a19c2499 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:03.555103 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a32607-dee7-4122-8a21-fbd2a19c2499 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:03.637075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 568/1134] 10.222.0.67 () {60 vars in 1141 bytes} [Fri Jun 24 15:02:03 2022] POST /identity/v3/projects => generated 311 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:02:05.690317 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd41246-abcf-44ef-98b8-cda456a78bec None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:05.690904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd41246-abcf-44ef-98b8-cda456a78bec None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:05.690904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd41246-abcf-44ef-98b8-cda456a78bec None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:05.692299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 567/1135] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:02:05.697471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a225e31-9729-45be-a5dc-dad4dbbaa5bb None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:05.697861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a225e31-9729-45be-a5dc-dad4dbbaa5bb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:05.698029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a225e31-9729-45be-a5dc-dad4dbbaa5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:05.714635 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3a225e31-9729-45be-a5dc-dad4dbbaa5bb None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:02:05.715822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3a225e31-9729-45be-a5dc-dad4dbbaa5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:02:05.778725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 569/1136] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:02:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:02:05.803637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd41246-abcf-44ef-98b8-cda456a78bec None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:05.872076 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:05.901592 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd41246-abcf-44ef-98b8-cda456a78bec None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:05.934133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974d823-a3a3-4592-a8b3-8f323fbebdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:05.934912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974d823-a3a3-4592-a8b3-8f323fbebdbd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:05.935216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974d823-a3a3-4592-a8b3-8f323fbebdbd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:05.935497 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974d823-a3a3-4592-a8b3-8f323fbebdbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:05.936659 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974d823-a3a3-4592-a8b3-8f323fbebdbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:05.938066 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974d823-a3a3-4592-a8b3-8f323fbebdbd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:05.940441 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 568/1137] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:02:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:05.945962 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a225e31-9729-45be-a5dc-dad4dbbaa5bb None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:05.995308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:06.040281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a225e31-9729-45be-a5dc-dad4dbbaa5bb None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:06.073763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241b929-cdbe-42ea-b76c-6a747dd80a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:06.074418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241b929-cdbe-42ea-b76c-6a747dd80a84 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:06.074700 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241b929-cdbe-42ea-b76c-6a747dd80a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:06.074975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241b929-cdbe-42ea-b76c-6a747dd80a84 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:06.075697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241b929-cdbe-42ea-b76c-6a747dd80a84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:06.077114 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241b929-cdbe-42ea-b76c-6a747dd80a84 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:06.184453 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 570/1138] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 15:02:05 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:02:08.229848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73976d-4036-4f58-a674-e1c1e0b4d719 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:08.230388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73976d-4036-4f58-a674-e1c1e0b4d719 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:08.230388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73976d-4036-4f58-a674-e1c1e0b4d719 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:08.231798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 569/1139] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:02:08.237647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e1cff-3752-49f5-b48e-f3bdc2aef071 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:08.238070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e1cff-3752-49f5-b48e-f3bdc2aef071 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:08.238201 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e1cff-3752-49f5-b48e-f3bdc2aef071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:08.254646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2f9e1cff-3752-49f5-b48e-f3bdc2aef071 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:02:08.255844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2f9e1cff-3752-49f5-b48e-f3bdc2aef071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:02:08.319987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 571/1140] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:02:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:02:08.343878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73976d-4036-4f58-a674-e1c1e0b4d719 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:08.441851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73976d-4036-4f58-a674-e1c1e0b4d719 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:08.475314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6fe8c-09dc-4a7b-9112-6cf956e6c1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:08.476096 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6fe8c-09dc-4a7b-9112-6cf956e6c1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:08.476382 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6fe8c-09dc-4a7b-9112-6cf956e6c1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:08.476662 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6fe8c-09dc-4a7b-9112-6cf956e6c1a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:08.484039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:08.488480 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6fe8c-09dc-4a7b-9112-6cf956e6c1a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-cdc6fe8c-09dc-4a7b-9112-6cf956e6c1a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:02:08.496140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:02:08.497137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:02:08.498148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:02:08.498148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:02:08.498148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:02:08.498148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:02:08.498148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:02:08.498148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:02:08.498148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:02:08.498148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:02:08.498148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:02:08.498148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:02:08.498148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:02:08.498148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:02:08.498148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 570/1141] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:02:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:02:08.503253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e1cff-3752-49f5-b48e-f3bdc2aef071 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:08.541911 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:08.598880 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e1cff-3752-49f5-b48e-f3bdc2aef071 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:08.630454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd4801-5957-4292-9baf-c7a6159d5b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:08.631110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd4801-5957-4292-9baf-c7a6159d5b58 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:08.631423 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd4801-5957-4292-9baf-c7a6159d5b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:08.631704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd4801-5957-4292-9baf-c7a6159d5b58 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:08.632441 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd4801-5957-4292-9baf-c7a6159d5b58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:02:08.632746 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd4801-5957-4292-9baf-c7a6159d5b58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:08.634100 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd4801-5957-4292-9baf-c7a6159d5b58 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:08.642386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 572/1142] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:02:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:08.648588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6fe8c-09dc-4a7b-9112-6cf956e6c1a8 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:08.746103 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6fe8c-09dc-4a7b-9112-6cf956e6c1a8 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:08.779247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80889e87-a00e-4454-ae43-e9c101403b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:08.780102 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80889e87-a00e-4454-ae43-e9c101403b59 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:08.780386 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80889e87-a00e-4454-ae43-e9c101403b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:08.780660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80889e87-a00e-4454-ae43-e9c101403b59 None admin] PATH_INFO: `/v3/users/fad3ca2056da4eafbd08bc6616cf3e75` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:08.794053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80889e87-a00e-4454-ae43-e9c101403b59 None admin] RBAC: Authorizing `identity:get_user(user_id=fad3ca2056da4eafbd08bc6616cf3e75)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:08.795510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80889e87-a00e-4454-ae43-e9c101403b59 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:08.797142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 571/1143] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:02:08 2022] GET /identity/v3/users/fad3ca2056da4eafbd08bc6616cf3e75 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:08.803275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd4801-5957-4292-9baf-c7a6159d5b58 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:08.899492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:08.902311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd4801-5957-4292-9baf-c7a6159d5b58 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:08.934584 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6dbf6-3636-4ab7-955b-49993e185edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:08.935244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6dbf6-3636-4ab7-955b-49993e185edb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:08.935532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6dbf6-3636-4ab7-955b-49993e185edb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:08.935808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6dbf6-3636-4ab7-955b-49993e185edb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:08.999492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6dbf6-3636-4ab7-955b-49993e185edb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:09.001537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6dbf6-3636-4ab7-955b-49993e185edb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:09.092229 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:09.096110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 573/1144] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:02:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:02:09.101900 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80889e87-a00e-4454-ae43-e9c101403b59 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:09.161984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:09.201578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80889e87-a00e-4454-ae43-e9c101403b59 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:09.233986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1f740-dd0d-444d-b041-fd962a515422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:09.234835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1f740-dd0d-444d-b041-fd962a515422 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:09.235626 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1f740-dd0d-444d-b041-fd962a515422 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:09.236122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1f740-dd0d-444d-b041-fd962a515422 None admin] PATH_INFO: `/v3/projects/8cc7618e5423482993d82b96840696db` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:09.237300 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1f740-dd0d-444d-b041-fd962a515422 None admin] RBAC: Authorizing `identity:get_project(project_id=8cc7618e5423482993d82b96840696db)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:09.238713 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1f740-dd0d-444d-b041-fd962a515422 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:09.240417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 572/1145] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:02:09 2022] GET /identity/v3/projects/8cc7618e5423482993d82b96840696db => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:09.246326 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6dbf6-3636-4ab7-955b-49993e185edb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:09.341344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6dbf6-3636-4ab7-955b-49993e185edb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:09.374482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4489139-7c03-40c9-a035-944530ab0670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:09.375122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4489139-7c03-40c9-a035-944530ab0670 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:09.375433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4489139-7c03-40c9-a035-944530ab0670 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:09.375708 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4489139-7c03-40c9-a035-944530ab0670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:09.377250 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4489139-7c03-40c9-a035-944530ab0670 None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=8cc7618e5423482993d82b96840696db, user.id=fad3ca2056da4eafbd08bc6616cf3e75) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:02:09.377565 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4489139-7c03-40c9-a035-944530ab0670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:09.378980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4489139-7c03-40c9-a035-944530ab0670 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:09.384268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 574/1146] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:02:09 2022] GET /identity/v3/role_assignments?user.id=fad3ca2056da4eafbd08bc6616cf3e75&scope.project.id=8cc7618e5423482993d82b96840696db&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:11.388863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa649284-01eb-40e4-bd62-65b317fad109 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:11.389399 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa649284-01eb-40e4-bd62-65b317fad109 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:11.389399 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa649284-01eb-40e4-bd62-65b317fad109 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:11.390760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 573/1147] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:02:11.396197 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ae133-d93c-4370-8d58-c9baa19001d8 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:11.396585 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ae133-d93c-4370-8d58-c9baa19001d8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:11.396760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ae133-d93c-4370-8d58-c9baa19001d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:11.413756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a58ae133-d93c-4370-8d58-c9baa19001d8 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:02:11.414934 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a58ae133-d93c-4370-8d58-c9baa19001d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:02:11.472669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:11.480688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 575/1148] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:02:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:02:11.503875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa649284-01eb-40e4-bd62-65b317fad109 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:11.601893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa649284-01eb-40e4-bd62-65b317fad109 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:11.625886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:11.634064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc1380-ac4e-4169-8423-89d500f31ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:11.637175 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc1380-ac4e-4169-8423-89d500f31ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:11.637175 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc1380-ac4e-4169-8423-89d500f31ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:11.637175 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc1380-ac4e-4169-8423-89d500f31ca7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:11.645646 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc1380-ac4e-4169-8423-89d500f31ca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-1adc1380-ac4e-4169-8423-89d500f31ca7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:02:11.653588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:02:11.654694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:02:11.655634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:02:11.655634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:02:11.655634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:02:11.655634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:02:11.655634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:02:11.655634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:02:11.655634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:02:11.655634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:02:11.655634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:02:11.655634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:02:11.655634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:02:11.655634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:02:11.655634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 574/1149] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:02:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:02:11.662007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ae133-d93c-4370-8d58-c9baa19001d8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:11.757940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ae133-d93c-4370-8d58-c9baa19001d8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:11.791124 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeef80c-43eb-4776-b62e-8da2602e720e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:11.791817 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeef80c-43eb-4776-b62e-8da2602e720e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:11.792104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeef80c-43eb-4776-b62e-8da2602e720e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:11.792380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeef80c-43eb-4776-b62e-8da2602e720e None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:11.793122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeef80c-43eb-4776-b62e-8da2602e720e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:02:11.793431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeef80c-43eb-4776-b62e-8da2602e720e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:11.794786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeef80c-43eb-4776-b62e-8da2602e720e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:11.803555 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 576/1150] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:02:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:11.809170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc1380-ac4e-4169-8423-89d500f31ca7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:11.906249 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc1380-ac4e-4169-8423-89d500f31ca7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:11.940717 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e0992-143d-4a18-84dc-3dfe82e3b83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:11.941569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e0992-143d-4a18-84dc-3dfe82e3b83d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:11.941851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e0992-143d-4a18-84dc-3dfe82e3b83d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:11.942124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e0992-143d-4a18-84dc-3dfe82e3b83d None admin] PATH_INFO: `/v3/users/fad3ca2056da4eafbd08bc6616cf3e75` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:11.943311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e0992-143d-4a18-84dc-3dfe82e3b83d None admin] RBAC: Authorizing `identity:get_user(user_id=fad3ca2056da4eafbd08bc6616cf3e75)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:11.944729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e0992-143d-4a18-84dc-3dfe82e3b83d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:11.946334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 575/1151] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:02:11 2022] GET /identity/v3/users/fad3ca2056da4eafbd08bc6616cf3e75 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:11.952170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeef80c-43eb-4776-b62e-8da2602e720e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:12.006705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:12.050004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeef80c-43eb-4776-b62e-8da2602e720e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:12.085048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efb246-76c6-4c90-b273-101c1a520d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:12.085909 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efb246-76c6-4c90-b273-101c1a520d1b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:12.086195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efb246-76c6-4c90-b273-101c1a520d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:12.086478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efb246-76c6-4c90-b273-101c1a520d1b None admin] PATH_INFO: `/v3/projects/8cc7618e5423482993d82b96840696db` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:12.087704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7efb246-76c6-4c90-b273-101c1a520d1b None admin] RBAC: Authorizing `identity:get_project(project_id=8cc7618e5423482993d82b96840696db)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:12.089135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7efb246-76c6-4c90-b273-101c1a520d1b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:12.093199 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 577/1152] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:02:11 2022] GET /identity/v3/projects/8cc7618e5423482993d82b96840696db => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:12.097096 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e0992-143d-4a18-84dc-3dfe82e3b83d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:12.149839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:12.193534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e0992-143d-4a18-84dc-3dfe82e3b83d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:12.225595 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bd545-bf4e-4eae-ab59-15594a078d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:12.226472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bd545-bf4e-4eae-ab59-15594a078d85 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:12.226750 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bd545-bf4e-4eae-ab59-15594a078d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:12.227021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bd545-bf4e-4eae-ab59-15594a078d85 None admin] PATH_INFO: `/v3/projects/8cc7618e5423482993d82b96840696db/users/fad3ca2056da4eafbd08bc6616cf3e75/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:12.228959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bd545-bf4e-4eae-ab59-15594a078d85 None admin] RBAC: Authorizing `identity:create_grant(project_id=8cc7618e5423482993d82b96840696db, user_id=fad3ca2056da4eafbd08bc6616cf3e75, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:12.230369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bd545-bf4e-4eae-ab59-15594a078d85 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:12.321389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 576/1153] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:02:12 2022] PUT /identity/v3/projects/8cc7618e5423482993d82b96840696db/users/fad3ca2056da4eafbd08bc6616cf3e75/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:02:14.332973 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c37eba-7c1a-4075-bbfe-335c542de449 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:14.333517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c37eba-7c1a-4075-bbfe-335c542de449 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:14.333517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c37eba-7c1a-4075-bbfe-335c542de449 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:14.334880 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 578/1154] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:02:14.340114 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a27ee3-19f3-4084-8965-034cb1d31de7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:14.340544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a27ee3-19f3-4084-8965-034cb1d31de7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:14.340656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a27ee3-19f3-4084-8965-034cb1d31de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:14.357472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a2a27ee3-19f3-4084-8965-034cb1d31de7 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:02:14.358573 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a2a27ee3-19f3-4084-8965-034cb1d31de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:02:14.421911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 577/1155] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:02:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:02:14.445771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c37eba-7c1a-4075-bbfe-335c542de449 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:14.544724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c37eba-7c1a-4075-bbfe-335c542de449 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:14.567110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:14.577167 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086ee41-26ec-4c81-8018-ad1216984fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:14.577949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086ee41-26ec-4c81-8018-ad1216984fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:14.578234 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086ee41-26ec-4c81-8018-ad1216984fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:14.582374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086ee41-26ec-4c81-8018-ad1216984fa7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:14.588728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086ee41-26ec-4c81-8018-ad1216984fa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f086ee41-26ec-4c81-8018-ad1216984fa7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:02:14.596978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:02:14.598084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:02:14.599133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:02:14.599133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:02:14.599133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:02:14.599133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:02:14.599133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:02:14.599133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:02:14.599133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:02:14.599133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:02:14.599133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:02:14.599133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:02:14.599133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:02:14.599133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:02:14.599133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 579/1156] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:02:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:02:14.604012 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a27ee3-19f3-4084-8965-034cb1d31de7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:14.647871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:14.700657 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a27ee3-19f3-4084-8965-034cb1d31de7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:14.735037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d50eaf-6629-44ed-9fb5-77a993721944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:14.735968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d50eaf-6629-44ed-9fb5-77a993721944 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:14.736252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d50eaf-6629-44ed-9fb5-77a993721944 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:14.736527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d50eaf-6629-44ed-9fb5-77a993721944 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:14.737266 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d50eaf-6629-44ed-9fb5-77a993721944 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:02:14.737574 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d50eaf-6629-44ed-9fb5-77a993721944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:14.738926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d50eaf-6629-44ed-9fb5-77a993721944 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:14.746858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 578/1157] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:02:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:14.752727 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086ee41-26ec-4c81-8018-ad1216984fa7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:14.850981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086ee41-26ec-4c81-8018-ad1216984fa7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:14.883074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091f448-fff6-42b2-adc1-8759b96ec48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:14.883948 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091f448-fff6-42b2-adc1-8759b96ec48d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:14.884228 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091f448-fff6-42b2-adc1-8759b96ec48d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:14.884499 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091f448-fff6-42b2-adc1-8759b96ec48d None admin] PATH_INFO: `/v3/users/fad3ca2056da4eafbd08bc6616cf3e75` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:14.885646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091f448-fff6-42b2-adc1-8759b96ec48d None admin] RBAC: Authorizing `identity:get_user(user_id=fad3ca2056da4eafbd08bc6616cf3e75)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:14.887061 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091f448-fff6-42b2-adc1-8759b96ec48d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:14.888702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 580/1158] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:02:14 2022] GET /identity/v3/users/fad3ca2056da4eafbd08bc6616cf3e75 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:14.895701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d50eaf-6629-44ed-9fb5-77a993721944 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:14.992385 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d50eaf-6629-44ed-9fb5-77a993721944 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:14.998936 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:15.026069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7b590-0bd6-416b-bb9d-a96c5888bb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:15.026700 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7b590-0bd6-416b-bb9d-a96c5888bb95 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:15.026979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7b590-0bd6-416b-bb9d-a96c5888bb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:15.027278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7b590-0bd6-416b-bb9d-a96c5888bb95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:15.093773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7b590-0bd6-416b-bb9d-a96c5888bb95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:15.095063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7b590-0bd6-416b-bb9d-a96c5888bb95 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:15.191145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 579/1159] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:02:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:02:15.196907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091f448-fff6-42b2-adc1-8759b96ec48d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:15.247901 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:15.292334 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091f448-fff6-42b2-adc1-8759b96ec48d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:15.324723 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbace8-d026-49de-999b-af5667ca9867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:15.325576 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbace8-d026-49de-999b-af5667ca9867 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:15.325857 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbace8-d026-49de-999b-af5667ca9867 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:15.326129 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbace8-d026-49de-999b-af5667ca9867 None admin] PATH_INFO: `/v3/projects/8cc7618e5423482993d82b96840696db` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:15.327302 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adbace8-d026-49de-999b-af5667ca9867 None admin] RBAC: Authorizing `identity:get_project(project_id=8cc7618e5423482993d82b96840696db)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:15.328734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adbace8-d026-49de-999b-af5667ca9867 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:15.330393 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 581/1160] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:02:15 2022] GET /identity/v3/projects/8cc7618e5423482993d82b96840696db => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:15.336308 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7b590-0bd6-416b-bb9d-a96c5888bb95 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:15.344204 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:15.431835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7b590-0bd6-416b-bb9d-a96c5888bb95 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:15.464889 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c200edb-9a2f-4bc3-bb43-591fbb737ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:15.465532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c200edb-9a2f-4bc3-bb43-591fbb737ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:15.465815 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c200edb-9a2f-4bc3-bb43-591fbb737ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:15.466090 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c200edb-9a2f-4bc3-bb43-591fbb737ed4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:15.466907 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c200edb-9a2f-4bc3-bb43-591fbb737ed4 None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=8cc7618e5423482993d82b96840696db, user.id=fad3ca2056da4eafbd08bc6616cf3e75) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:02:15.467240 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c200edb-9a2f-4bc3-bb43-591fbb737ed4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:15.468661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c200edb-9a2f-4bc3-bb43-591fbb737ed4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:15.474345 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 580/1161] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:02:15 2022] GET /identity/v3/role_assignments?user.id=fad3ca2056da4eafbd08bc6616cf3e75&scope.project.id=8cc7618e5423482993d82b96840696db&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:17.507759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf7277-b495-49f3-a757-b6d5bf1988d3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:17.508292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf7277-b495-49f3-a757-b6d5bf1988d3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:17.508292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf7277-b495-49f3-a757-b6d5bf1988d3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:17.509677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 582/1162] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:02:17.514828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3924bf-a353-451a-b155-38c7cb8fe3d1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:17.515220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3924bf-a353-451a-b155-38c7cb8fe3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:17.515411 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3924bf-a353-451a-b155-38c7cb8fe3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:17.532651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-aa3924bf-a353-451a-b155-38c7cb8fe3d1 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:02:17.533989 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-aa3924bf-a353-451a-b155-38c7cb8fe3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:02:17.593958 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:17.598986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 581/1163] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:02:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:02:17.621736 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf7277-b495-49f3-a757-b6d5bf1988d3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:17.719995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf7277-b495-49f3-a757-b6d5bf1988d3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:17.739341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:17.752436 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e2de4f-66c0-4f3a-b081-4dba9cb1cf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:17.753075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2de4f-66c0-4f3a-b081-4dba9cb1cf1b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:17.754333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2de4f-66c0-4f3a-b081-4dba9cb1cf1b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:17.754333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2de4f-66c0-4f3a-b081-4dba9cb1cf1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:17.818462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e2de4f-66c0-4f3a-b081-4dba9cb1cf1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:17.819863 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e2de4f-66c0-4f3a-b081-4dba9cb1cf1b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:17.914284 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 583/1164] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:02:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:02:17.920319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3924bf-a353-451a-b155-38c7cb8fe3d1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:18.032814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3924bf-a353-451a-b155-38c7cb8fe3d1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:18.075366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db27d15-91d4-43f8-9cf1-0cae45c8f8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:18.076106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db27d15-91d4-43f8-9cf1-0cae45c8f8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:18.076391 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db27d15-91d4-43f8-9cf1-0cae45c8f8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:18.076669 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db27d15-91d4-43f8-9cf1-0cae45c8f8e6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:18.083567 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db27d15-91d4-43f8-9cf1-0cae45c8f8e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:18.085037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db27d15-91d4-43f8-9cf1-0cae45c8f8e6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-0db27d15-91d4-43f8-9cf1-0cae45c8f8e6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:02:18.100074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:02:18.101320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:02:18.102532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:02:18.102532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:02:18.102532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:02:18.102532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:02:18.102532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:02:18.102532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:02:18.102532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:02:18.102532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:02:18.102532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:02:18.102532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:02:18.102532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 24 15:02:18.102532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:02:18.102532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 15:02:18.102532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:02:18.102532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 582/1165] 10.222.0.67 () {56 vars in 1117 bytes} [Fri Jun 24 15:02:17 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:02:18.107929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e2de4f-66c0-4f3a-b081-4dba9cb1cf1b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:18.129674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:18.204144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e2de4f-66c0-4f3a-b081-4dba9cb1cf1b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:18.237140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f386d31-02d9-4f48-a7a3-4bc6e9c72bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:18.237791 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f386d31-02d9-4f48-a7a3-4bc6e9c72bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:18.238074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f386d31-02d9-4f48-a7a3-4bc6e9c72bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:18.238350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f386d31-02d9-4f48-a7a3-4bc6e9c72bc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:18.239067 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f386d31-02d9-4f48-a7a3-4bc6e9c72bc0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:02:18.239399 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f386d31-02d9-4f48-a7a3-4bc6e9c72bc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:18.240759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f386d31-02d9-4f48-a7a3-4bc6e9c72bc0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:18.247664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 584/1166] 10.222.0.67 () {56 vars in 1115 bytes} [Fri Jun 24 15:02:18 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:20.252789 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940b36c-bc45-4c20-914d-6a32d0927b37 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:20.253311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940b36c-bc45-4c20-914d-6a32d0927b37 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:20.253430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940b36c-bc45-4c20-914d-6a32d0927b37 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:20.254711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 583/1167] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:02:20.259872 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c52df-67b7-4511-b7f4-247f3ae276cf None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:20.260278 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c52df-67b7-4511-b7f4-247f3ae276cf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:20.260420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c52df-67b7-4511-b7f4-247f3ae276cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:20.277155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-536c52df-67b7-4511-b7f4-247f3ae276cf None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:02:20.278316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-536c52df-67b7-4511-b7f4-247f3ae276cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:02:20.341863 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 585/1168] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:02:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:02:20.364899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940b36c-bc45-4c20-914d-6a32d0927b37 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:20.424114 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:20.465063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940b36c-bc45-4c20-914d-6a32d0927b37 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:20.497224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a656d37-6f5f-4616-9b5c-6a0fe7e120ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:20.497871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a656d37-6f5f-4616-9b5c-6a0fe7e120ad None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:20.498154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a656d37-6f5f-4616-9b5c-6a0fe7e120ad None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:20.498429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a656d37-6f5f-4616-9b5c-6a0fe7e120ad None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:20.499169 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a656d37-6f5f-4616-9b5c-6a0fe7e120ad None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:20.501215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a656d37-6f5f-4616-9b5c-6a0fe7e120ad None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:20.602662 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 584/1169] 10.222.0.67 () {60 vars in 1138 bytes} [Fri Jun 24 15:02:20 2022] POST /identity/v3/domains => generated 267 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:02:22.651208 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a3f0f-c994-458a-8a6b-a8df53fb86cf None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:22.651780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a3f0f-c994-458a-8a6b-a8df53fb86cf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:22.651780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a3f0f-c994-458a-8a6b-a8df53fb86cf None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:22.653140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 586/1170] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:02:22.658443 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9deeed-413c-49e6-994d-776408d7555e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:22.658881 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9deeed-413c-49e6-994d-776408d7555e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:22.658987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9deeed-413c-49e6-994d-776408d7555e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:22.675808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3c9deeed-413c-49e6-994d-776408d7555e None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:02:22.676986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3c9deeed-413c-49e6-994d-776408d7555e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:02:22.742042 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 585/1171] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:02:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:02:22.766824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a3f0f-c994-458a-8a6b-a8df53fb86cf None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:22.782542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:22.865847 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a3f0f-c994-458a-8a6b-a8df53fb86cf None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:22.898612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a615dfa-5be8-49b0-a662-1b4468ac5cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:22.899500 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a615dfa-5be8-49b0-a662-1b4468ac5cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:22.899780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a615dfa-5be8-49b0-a662-1b4468ac5cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:22.900053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a615dfa-5be8-49b0-a662-1b4468ac5cc2 None admin] PATH_INFO: `/v3/domains/8289b7bb2cc94359a40844b54a18e78f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:22.910890 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a615dfa-5be8-49b0-a662-1b4468ac5cc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8289b7bb2cc94359a40844b54a18e78f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:22.912288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a615dfa-5be8-49b0-a662-1b4468ac5cc2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:22.913893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 587/1172] 10.222.0.67 () {56 vars in 1183 bytes} [Fri Jun 24 15:02:22 2022] GET /identity/v3/domains/8289b7bb2cc94359a40844b54a18e78f => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:22.919725 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9deeed-413c-49e6-994d-776408d7555e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:22.968801 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:23.017536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9deeed-413c-49e6-994d-776408d7555e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:23.051388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389fa6a-8af3-44c7-8b5d-93e5faae3975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:23.052041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389fa6a-8af3-44c7-8b5d-93e5faae3975 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:23.052325 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389fa6a-8af3-44c7-8b5d-93e5faae3975 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:23.052602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389fa6a-8af3-44c7-8b5d-93e5faae3975 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:23.053298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389fa6a-8af3-44c7-8b5d-93e5faae3975 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:23.054718 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389fa6a-8af3-44c7-8b5d-93e5faae3975 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:23.129302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 586/1173] 10.222.0.67 () {60 vars in 1141 bytes} [Fri Jun 24 15:02:22 2022] POST /identity/v3/projects => generated 361 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:02:25.151945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6aed2c-4635-4a8a-8cc1-c72a0508ee99 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:25.152503 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6aed2c-4635-4a8a-8cc1-c72a0508ee99 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:25.152503 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6aed2c-4635-4a8a-8cc1-c72a0508ee99 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:25.153890 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 588/1174] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:02:25.159504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d64d3f-55b5-4b62-869c-2471df07cd74 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:25.159939 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d64d3f-55b5-4b62-869c-2471df07cd74 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:25.160095 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d64d3f-55b5-4b62-869c-2471df07cd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:25.176748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-81d64d3f-55b5-4b62-869c-2471df07cd74 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:02:25.177924 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-81d64d3f-55b5-4b62-869c-2471df07cd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:02:25.242523 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 587/1175] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:02:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 15:02:25.267558 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6aed2c-4635-4a8a-8cc1-c72a0508ee99 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:25.342043 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:25.367261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6aed2c-4635-4a8a-8cc1-c72a0508ee99 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:25.401472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb424f1-0744-46fa-82fc-2956860f37f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:25.402324 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb424f1-0744-46fa-82fc-2956860f37f8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:25.402611 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb424f1-0744-46fa-82fc-2956860f37f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:25.402891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb424f1-0744-46fa-82fc-2956860f37f8 None admin] PATH_INFO: `/v3/domains/8289b7bb2cc94359a40844b54a18e78f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:25.404095 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb424f1-0744-46fa-82fc-2956860f37f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8289b7bb2cc94359a40844b54a18e78f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:25.405518 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb424f1-0744-46fa-82fc-2956860f37f8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:25.407503 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 589/1176] 10.222.0.67 () {56 vars in 1183 bytes} [Fri Jun 24 15:02:25 2022] GET /identity/v3/domains/8289b7bb2cc94359a40844b54a18e78f => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:02:25.413169 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d64d3f-55b5-4b62-869c-2471df07cd74 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:25.443100 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:25.513397 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d64d3f-55b5-4b62-869c-2471df07cd74 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:25.546395 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c77c4-df7b-4c3c-8192-eea04851f019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:25.547122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c77c4-df7b-4c3c-8192-eea04851f019 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:25.547512 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c77c4-df7b-4c3c-8192-eea04851f019 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:25.547850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c77c4-df7b-4c3c-8192-eea04851f019 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:25.548562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c77c4-df7b-4c3c-8192-eea04851f019 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:25.549977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c77c4-df7b-4c3c-8192-eea04851f019 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:25.630522 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 588/1177] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 15:02:25 2022] POST /identity/v3/users => generated 311 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:02:27.724428 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660147b2-2f7c-4699-8384-ce811164a902 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:27.724986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660147b2-2f7c-4699-8384-ce811164a902 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:27.724986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660147b2-2f7c-4699-8384-ce811164a902 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:27.726363 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 590/1178] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:02:27.731481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882f930-4df5-410d-927e-2192ac1ce3b2 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:27.731921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882f930-4df5-410d-927e-2192ac1ce3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:27.732017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882f930-4df5-410d-927e-2192ac1ce3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:27.748875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f882f930-4df5-410d-927e-2192ac1ce3b2 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:02:27.750653 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f882f930-4df5-410d-927e-2192ac1ce3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:02:27.815089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 589/1179] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:02:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:02:27.838521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660147b2-2f7c-4699-8384-ce811164a902 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:27.933895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660147b2-2f7c-4699-8384-ce811164a902 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:27.968219 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1b636-11ca-4411-ad89-0d3a35900e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:27.969093 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1b636-11ca-4411-ad89-0d3a35900e9b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:27.969403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1b636-11ca-4411-ad89-0d3a35900e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:27.969749 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1b636-11ca-4411-ad89-0d3a35900e9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:27.979549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1b636-11ca-4411-ad89-0d3a35900e9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-79a1b636-11ca-4411-ad89-0d3a35900e9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:02:27.987663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:02:27.988840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:02:27.989950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:02:27.989950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:02:27.989950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:02:27.989950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:02:27.989950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:02:27.989950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:02:27.989950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:02:27.989950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:02:27.989950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:02:27.989950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:02:27.989950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:02:27.989950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:02:27.989950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 591/1180] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:02:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:02:27.994844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882f930-4df5-410d-927e-2192ac1ce3b2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:28.013512 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:28.093880 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882f930-4df5-410d-927e-2192ac1ce3b2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:28.128609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2cfa0-3d52-4182-a244-3ab642917691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:28.129278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2cfa0-3d52-4182-a244-3ab642917691 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:28.129560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2cfa0-3d52-4182-a244-3ab642917691 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:28.129834 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2cfa0-3d52-4182-a244-3ab642917691 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:28.130578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2cfa0-3d52-4182-a244-3ab642917691 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:02:28.130885 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2cfa0-3d52-4182-a244-3ab642917691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:28.132269 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2cfa0-3d52-4182-a244-3ab642917691 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:28.140372 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 590/1181] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:02:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:28.146534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1b636-11ca-4411-ad89-0d3a35900e9b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:28.149342 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:28.243155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1b636-11ca-4411-ad89-0d3a35900e9b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:28.275724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e9750-75e3-49cb-8fab-3a346bad1c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:28.276583 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e9750-75e3-49cb-8fab-3a346bad1c2b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:28.276865 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e9750-75e3-49cb-8fab-3a346bad1c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:28.277139 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e9750-75e3-49cb-8fab-3a346bad1c2b None admin] PATH_INFO: `/v3/users/3ffd0655d816433b9b5c2763f53b1c24` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:28.289641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e9750-75e3-49cb-8fab-3a346bad1c2b None admin] RBAC: Authorizing `identity:get_user(user_id=3ffd0655d816433b9b5c2763f53b1c24)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:28.291069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e9750-75e3-49cb-8fab-3a346bad1c2b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:28.292716 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 592/1182] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:02:28 2022] GET /identity/v3/users/3ffd0655d816433b9b5c2763f53b1c24 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:28.298888 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2cfa0-3d52-4182-a244-3ab642917691 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:28.376765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:28.394454 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2cfa0-3d52-4182-a244-3ab642917691 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:28.427589 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa830a3-3e6c-4d96-84c1-69dd2867d63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:28.428225 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa830a3-3e6c-4d96-84c1-69dd2867d63b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:28.428508 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa830a3-3e6c-4d96-84c1-69dd2867d63b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:28.428785 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa830a3-3e6c-4d96-84c1-69dd2867d63b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:28.492910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa830a3-3e6c-4d96-84c1-69dd2867d63b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:28.494386 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa830a3-3e6c-4d96-84c1-69dd2867d63b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:28.569850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:28.588360 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 591/1183] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:02:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:02:28.594838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e9750-75e3-49cb-8fab-3a346bad1c2b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:28.650857 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:28.690490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e9750-75e3-49cb-8fab-3a346bad1c2b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:28.722203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cd191-f28f-49d6-940f-c680f775fa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:28.723056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cd191-f28f-49d6-940f-c680f775fa92 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:28.723368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cd191-f28f-49d6-940f-c680f775fa92 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:28.723647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cd191-f28f-49d6-940f-c680f775fa92 None admin] PATH_INFO: `/v3/projects/844de276ff4549ba87f08266806a832b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:28.724806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66cd191-f28f-49d6-940f-c680f775fa92 None admin] RBAC: Authorizing `identity:get_project(project_id=844de276ff4549ba87f08266806a832b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:28.726232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66cd191-f28f-49d6-940f-c680f775fa92 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:28.727930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 593/1184] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:02:28 2022] GET /identity/v3/projects/844de276ff4549ba87f08266806a832b => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:28.733770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa830a3-3e6c-4d96-84c1-69dd2867d63b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:28.829199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa830a3-3e6c-4d96-84c1-69dd2867d63b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:28.861757 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184bae1-8893-4f18-ba8a-7e87c8433f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:28.862400 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184bae1-8893-4f18-ba8a-7e87c8433f5c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:28.862680 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184bae1-8893-4f18-ba8a-7e87c8433f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:28.862954 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184bae1-8893-4f18-ba8a-7e87c8433f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:28.863812 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184bae1-8893-4f18-ba8a-7e87c8433f5c None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=844de276ff4549ba87f08266806a832b, user.id=3ffd0655d816433b9b5c2763f53b1c24) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:02:28.864126 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184bae1-8893-4f18-ba8a-7e87c8433f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:28.865540 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184bae1-8893-4f18-ba8a-7e87c8433f5c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:28.870750 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 592/1185] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:02:28 2022] GET /identity/v3/role_assignments?user.id=3ffd0655d816433b9b5c2763f53b1c24&scope.project.id=844de276ff4549ba87f08266806a832b&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:30.860549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd295371-4a59-43ff-b7c3-e03a17328935 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:30.861122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd295371-4a59-43ff-b7c3-e03a17328935 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:30.861122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd295371-4a59-43ff-b7c3-e03a17328935 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:30.862449 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 594/1186] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:02:30.867581 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96c57c-15d0-4faa-a44c-828e4165caf5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:30.868000 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96c57c-15d0-4faa-a44c-828e4165caf5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:30.868134 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96c57c-15d0-4faa-a44c-828e4165caf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:30.885317 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ce96c57c-15d0-4faa-a44c-828e4165caf5 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:02:30.886482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ce96c57c-15d0-4faa-a44c-828e4165caf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:02:30.926258 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:30.952205 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 593/1187] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:02:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:02:30.975378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd295371-4a59-43ff-b7c3-e03a17328935 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:31.075094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd295371-4a59-43ff-b7c3-e03a17328935 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:31.100683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:31.108202 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129864d-809b-4d71-99e7-cfca8292d43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:31.108993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129864d-809b-4d71-99e7-cfca8292d43e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:31.109803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129864d-809b-4d71-99e7-cfca8292d43e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:31.127792 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129864d-809b-4d71-99e7-cfca8292d43e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:31.127792 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129864d-809b-4d71-99e7-cfca8292d43e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4129864d-809b-4d71-99e7-cfca8292d43e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:02:31.128115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:02:31.129210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:02:31.130267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:02:31.130267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:02:31.130267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:02:31.130267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:02:31.130267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:02:31.130267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:02:31.130267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:02:31.130267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:02:31.130267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:02:31.130267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:02:31.130267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:02:31.130267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:02:31.130267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 595/1188] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:02:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:02:31.135368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96c57c-15d0-4faa-a44c-828e4165caf5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:31.230158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96c57c-15d0-4faa-a44c-828e4165caf5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:31.263305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b8ae8-8aa2-4107-b4b9-eddbad4ee817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:31.263967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b8ae8-8aa2-4107-b4b9-eddbad4ee817 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:31.264249 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b8ae8-8aa2-4107-b4b9-eddbad4ee817 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:31.264522 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b8ae8-8aa2-4107-b4b9-eddbad4ee817 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:31.265267 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b8ae8-8aa2-4107-b4b9-eddbad4ee817 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:02:31.265632 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b8ae8-8aa2-4107-b4b9-eddbad4ee817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:31.267141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b8ae8-8aa2-4107-b4b9-eddbad4ee817 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:31.274928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 594/1189] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:02:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:31.280538 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129864d-809b-4d71-99e7-cfca8292d43e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:31.375699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129864d-809b-4d71-99e7-cfca8292d43e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:31.407352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25320054-995c-4345-9847-4bdf62043fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:31.408202 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25320054-995c-4345-9847-4bdf62043fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:31.408478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25320054-995c-4345-9847-4bdf62043fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:31.408746 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25320054-995c-4345-9847-4bdf62043fa2 None admin] PATH_INFO: `/v3/users/3ffd0655d816433b9b5c2763f53b1c24` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:31.409893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25320054-995c-4345-9847-4bdf62043fa2 None admin] RBAC: Authorizing `identity:get_user(user_id=3ffd0655d816433b9b5c2763f53b1c24)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:31.411332 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25320054-995c-4345-9847-4bdf62043fa2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:31.412922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 596/1190] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:02:31 2022] GET /identity/v3/users/3ffd0655d816433b9b5c2763f53b1c24 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:31.418529 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b8ae8-8aa2-4107-b4b9-eddbad4ee817 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:31.454363 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:31.516449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b8ae8-8aa2-4107-b4b9-eddbad4ee817 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:31.548392 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ae08c-5745-4951-b25e-c44fe5b0a260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:31.549241 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ae08c-5745-4951-b25e-c44fe5b0a260 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:31.549526 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ae08c-5745-4951-b25e-c44fe5b0a260 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:31.549804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ae08c-5745-4951-b25e-c44fe5b0a260 None admin] PATH_INFO: `/v3/projects/844de276ff4549ba87f08266806a832b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:31.551250 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ae08c-5745-4951-b25e-c44fe5b0a260 None admin] RBAC: Authorizing `identity:get_project(project_id=844de276ff4549ba87f08266806a832b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:31.552663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ae08c-5745-4951-b25e-c44fe5b0a260 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:31.554326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 595/1191] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:02:31 2022] GET /identity/v3/projects/844de276ff4549ba87f08266806a832b => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:31.560251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25320054-995c-4345-9847-4bdf62043fa2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:31.614624 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:31.657029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25320054-995c-4345-9847-4bdf62043fa2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:31.689257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507345ae-901a-449f-b933-a82e007296b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:31.690074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507345ae-901a-449f-b933-a82e007296b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:31.690356 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507345ae-901a-449f-b933-a82e007296b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:31.690629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507345ae-901a-449f-b933-a82e007296b3 None admin] PATH_INFO: `/v3/projects/844de276ff4549ba87f08266806a832b/users/3ffd0655d816433b9b5c2763f53b1c24/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:31.692577 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507345ae-901a-449f-b933-a82e007296b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=844de276ff4549ba87f08266806a832b, user_id=3ffd0655d816433b9b5c2763f53b1c24, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:31.693995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507345ae-901a-449f-b933-a82e007296b3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:31.788030 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 597/1192] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:02:31 2022] PUT /identity/v3/projects/844de276ff4549ba87f08266806a832b/users/3ffd0655d816433b9b5c2763f53b1c24/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:02:33.763072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29483f9f-ee13-47c4-805b-414d345abe1d None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:33.763652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29483f9f-ee13-47c4-805b-414d345abe1d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:33.763652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29483f9f-ee13-47c4-805b-414d345abe1d None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:33.765028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 596/1193] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:02:33.770724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c88e5c-23da-4651-b86e-e7abfae3d135 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:33.771123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c88e5c-23da-4651-b86e-e7abfae3d135 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:33.771325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c88e5c-23da-4651-b86e-e7abfae3d135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:33.788182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e6c88e5c-23da-4651-b86e-e7abfae3d135 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:02:33.789356 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e6c88e5c-23da-4651-b86e-e7abfae3d135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:02:33.853976 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 598/1194] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:02:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:02:33.877940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29483f9f-ee13-47c4-805b-414d345abe1d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:33.976570 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29483f9f-ee13-47c4-805b-414d345abe1d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:33.978827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:34.009259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c0e94-6db4-47d0-bd15-4c1d7f63d082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:34.010030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c0e94-6db4-47d0-bd15-4c1d7f63d082 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:34.010314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c0e94-6db4-47d0-bd15-4c1d7f63d082 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:34.010654 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c0e94-6db4-47d0-bd15-4c1d7f63d082 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:34.021812 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c0e94-6db4-47d0-bd15-4c1d7f63d082 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-562c0e94-6db4-47d0-bd15-4c1d7f63d082 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:02:34.029494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:02:34.030618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:02:34.032349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:02:34.032349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:02:34.032349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:02:34.032349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:02:34.032349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:02:34.032349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:02:34.032349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:02:34.032349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:02:34.032349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:02:34.032349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:02:34.032349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:02:34.032349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:02:34.032349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 597/1195] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:02:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:02:34.037707 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c88e5c-23da-4651-b86e-e7abfae3d135 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:34.083281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:34.136024 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c88e5c-23da-4651-b86e-e7abfae3d135 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:34.169836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2735b-6314-47f3-a696-e60da55ab7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:34.170498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2735b-6314-47f3-a696-e60da55ab7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:34.170786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2735b-6314-47f3-a696-e60da55ab7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:34.171067 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2735b-6314-47f3-a696-e60da55ab7a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:34.171877 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2735b-6314-47f3-a696-e60da55ab7a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:02:34.172190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2735b-6314-47f3-a696-e60da55ab7a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:34.173559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2735b-6314-47f3-a696-e60da55ab7a8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:34.182003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 599/1196] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:02:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:34.187999 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c0e94-6db4-47d0-bd15-4c1d7f63d082 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:34.284118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c0e94-6db4-47d0-bd15-4c1d7f63d082 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:34.316145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e1b03-9950-48e4-89de-77051c853e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:34.317019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e1b03-9950-48e4-89de-77051c853e82 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:34.317300 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e1b03-9950-48e4-89de-77051c853e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:34.317569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e1b03-9950-48e4-89de-77051c853e82 None admin] PATH_INFO: `/v3/users/3ffd0655d816433b9b5c2763f53b1c24` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:34.318720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e1b03-9950-48e4-89de-77051c853e82 None admin] RBAC: Authorizing `identity:get_user(user_id=3ffd0655d816433b9b5c2763f53b1c24)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:34.320152 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e1b03-9950-48e4-89de-77051c853e82 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:34.321735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 598/1197] 10.222.0.67 () {56 vars in 1177 bytes} [Fri Jun 24 15:02:34 2022] GET /identity/v3/users/3ffd0655d816433b9b5c2763f53b1c24 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:34.327465 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2735b-6314-47f3-a696-e60da55ab7a8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:34.425727 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2735b-6314-47f3-a696-e60da55ab7a8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:34.434433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:34.459147 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d244321-0124-43f1-a14a-2978944066dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:34.459815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d244321-0124-43f1-a14a-2978944066dd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:34.460101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d244321-0124-43f1-a14a-2978944066dd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:34.460378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d244321-0124-43f1-a14a-2978944066dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:34.524562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d244321-0124-43f1-a14a-2978944066dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:34.526001 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d244321-0124-43f1-a14a-2978944066dd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:34.619530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 600/1198] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:02:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:02:34.625780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e1b03-9950-48e4-89de-77051c853e82 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:34.656304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:34.723478 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e1b03-9950-48e4-89de-77051c853e82 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:34.757677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b0d66-7cf8-42f9-88ed-1c7f3a179319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:34.758515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b0d66-7cf8-42f9-88ed-1c7f3a179319 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:34.758801 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b0d66-7cf8-42f9-88ed-1c7f3a179319 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:34.759076 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b0d66-7cf8-42f9-88ed-1c7f3a179319 None admin] PATH_INFO: `/v3/projects/844de276ff4549ba87f08266806a832b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:34.760292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b0d66-7cf8-42f9-88ed-1c7f3a179319 None admin] RBAC: Authorizing `identity:get_project(project_id=844de276ff4549ba87f08266806a832b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:34.761709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b0d66-7cf8-42f9-88ed-1c7f3a179319 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:34.763416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 599/1199] 10.222.0.67 () {56 vars in 1186 bytes} [Fri Jun 24 15:02:34 2022] GET /identity/v3/projects/844de276ff4549ba87f08266806a832b => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:34.769288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d244321-0124-43f1-a14a-2978944066dd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:34.779068 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:34.864972 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d244321-0124-43f1-a14a-2978944066dd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:34.897731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed52cf4-ffef-46f6-aa3d-336df6c2622b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:34.898371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed52cf4-ffef-46f6-aa3d-336df6c2622b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:34.898658 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed52cf4-ffef-46f6-aa3d-336df6c2622b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:34.898935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed52cf4-ffef-46f6-aa3d-336df6c2622b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:34.899794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed52cf4-ffef-46f6-aa3d-336df6c2622b None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=844de276ff4549ba87f08266806a832b, user.id=3ffd0655d816433b9b5c2763f53b1c24) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:02:34.900112 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed52cf4-ffef-46f6-aa3d-336df6c2622b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:34.901539 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed52cf4-ffef-46f6-aa3d-336df6c2622b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:34.907407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 601/1200] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:02:34 2022] GET /identity/v3/role_assignments?user.id=3ffd0655d816433b9b5c2763f53b1c24&scope.project.id=844de276ff4549ba87f08266806a832b&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:02:44.753563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=50 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:44.904639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=64 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:46.369433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad78ab-ea36-4a7c-bd9d-28a0f42adde0 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:46.369987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad78ab-ea36-4a7c-bd9d-28a0f42adde0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:46.369987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad78ab-ea36-4a7c-bd9d-28a0f42adde0 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:46.371372 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 600/1201] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:03:46.376802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d30aa-ff1c-468f-b3fd-4d5fdc9b301d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:46.377267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d30aa-ff1c-468f-b3fd-4d5fdc9b301d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:46.377370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d30aa-ff1c-468f-b3fd-4d5fdc9b301d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:46.393779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-895d30aa-ff1c-468f-b3fd-4d5fdc9b301d None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:03:46.399826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-895d30aa-ff1c-468f-b3fd-4d5fdc9b301d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:03:46.459997 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 602/1202] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:03:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:03:46.486044 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad78ab-ea36-4a7c-bd9d-28a0f42adde0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:46.589112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad78ab-ea36-4a7c-bd9d-28a0f42adde0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:46.625037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec9f12-9087-4805-b1a7-af8d39a3105c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:46.625796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec9f12-9087-4805-b1a7-af8d39a3105c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:46.626074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec9f12-9087-4805-b1a7-af8d39a3105c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:46.626351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec9f12-9087-4805-b1a7-af8d39a3105c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:46.636412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec9f12-9087-4805-b1a7-af8d39a3105c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:46.637837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec9f12-9087-4805-b1a7-af8d39a3105c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:46.655515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-4eec9f12-9087-4805-b1a7-af8d39a3105c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:03:46.655515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:46.655515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:46.655515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:46.655515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:46.655515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:46.655515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:46.655515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:46.655515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:46.655515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:46.655515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:46.655515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:46.655515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:03:46.655515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:03:46.655515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:46.659154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:46.661315 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:03:46.661315 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:03:46.661315 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:03:46.661315 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:03:46.661315 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 601/1203] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:03:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:46.664980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d30aa-ff1c-468f-b3fd-4d5fdc9b301d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:46.762586 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d30aa-ff1c-468f-b3fd-4d5fdc9b301d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:46.791704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:46.797419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69027c4-e3ed-46ad-92a7-a75ac4f24814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:46.798331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69027c4-e3ed-46ad-92a7-a75ac4f24814 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:46.798649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69027c4-e3ed-46ad-92a7-a75ac4f24814 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:46.798934 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69027c4-e3ed-46ad-92a7-a75ac4f24814 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:46.799688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69027c4-e3ed-46ad-92a7-a75ac4f24814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:46.799998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69027c4-e3ed-46ad-92a7-a75ac4f24814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:46.801355 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69027c4-e3ed-46ad-92a7-a75ac4f24814 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:46.812173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 603/1204] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:46.818216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec9f12-9087-4805-b1a7-af8d39a3105c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:46.870662 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:46.916082 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec9f12-9087-4805-b1a7-af8d39a3105c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:46.948864 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f7672-e9f5-4b4d-8ba7-70f6d88841cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:46.949525 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f7672-e9f5-4b4d-8ba7-70f6d88841cd None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:46.949805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f7672-e9f5-4b4d-8ba7-70f6d88841cd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:46.950079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f7672-e9f5-4b4d-8ba7-70f6d88841cd None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:46.950766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f7672-e9f5-4b4d-8ba7-70f6d88841cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:46.952212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f7672-e9f5-4b4d-8ba7-70f6d88841cd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:47.039833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 602/1205] 10.222.0.67 () {60 vars in 1132 bytes} [Fri Jun 24 15:03:46 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:03:49.074920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45e300-c2b6-48c2-8e23-a22bf40cc1cb None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:49.075544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45e300-c2b6-48c2-8e23-a22bf40cc1cb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:49.075544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45e300-c2b6-48c2-8e23-a22bf40cc1cb None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:49.076948 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 604/1206] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:03:49.082134 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403c249-5e38-40d3-a40a-0cd40e5dfe34 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:49.082557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403c249-5e38-40d3-a40a-0cd40e5dfe34 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:49.082696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403c249-5e38-40d3-a40a-0cd40e5dfe34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:49.101197 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4403c249-5e38-40d3-a40a-0cd40e5dfe34 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:03:49.102387 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4403c249-5e38-40d3-a40a-0cd40e5dfe34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:03:49.168123 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 603/1207] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:03:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:03:49.193288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45e300-c2b6-48c2-8e23-a22bf40cc1cb None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:49.294491 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45e300-c2b6-48c2-8e23-a22bf40cc1cb None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:49.327956 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3574f-aa7f-4e5d-ada8-407de3ab77db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:49.328709 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3574f-aa7f-4e5d-ada8-407de3ab77db None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:49.328990 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3574f-aa7f-4e5d-ada8-407de3ab77db None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:49.329264 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3574f-aa7f-4e5d-ada8-407de3ab77db None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:49.339261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3574f-aa7f-4e5d-ada8-407de3ab77db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-69e3574f-aa7f-4e5d-ada8-407de3ab77db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:03:49.347542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:49.351451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:49.352527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:49.352527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:49.352527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:49.352527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:49.352527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:03:49.352527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:03:49.352527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:03:49.352527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:03:49.352527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:03:49.352527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:03:49.352527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:03:49.352527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:03:49.352527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 605/1208] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:03:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:49.354695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403c249-5e38-40d3-a40a-0cd40e5dfe34 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:49.393764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:49.451248 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403c249-5e38-40d3-a40a-0cd40e5dfe34 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:49.484388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3013a4-be3b-4958-92dc-2ce0cb4b54f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:49.485733 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3013a4-be3b-4958-92dc-2ce0cb4b54f2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:49.486076 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3013a4-be3b-4958-92dc-2ce0cb4b54f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:49.486359 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3013a4-be3b-4958-92dc-2ce0cb4b54f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:49.487103 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3013a4-be3b-4958-92dc-2ce0cb4b54f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:49.487444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3013a4-be3b-4958-92dc-2ce0cb4b54f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:49.488802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3013a4-be3b-4958-92dc-2ce0cb4b54f2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:49.496541 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 604/1209] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:03:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:49.502669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3574f-aa7f-4e5d-ada8-407de3ab77db None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:49.553666 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:49.603280 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3574f-aa7f-4e5d-ada8-407de3ab77db None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:49.636456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0a27c-8d2d-493b-ac0a-df36fb68699a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:49.637250 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0a27c-8d2d-493b-ac0a-df36fb68699a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:49.637537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0a27c-8d2d-493b-ac0a-df36fb68699a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:49.637816 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0a27c-8d2d-493b-ac0a-df36fb68699a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:49.649697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0a27c-8d2d-493b-ac0a-df36fb68699a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:49.651135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0a27c-8d2d-493b-ac0a-df36fb68699a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:49.669660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-20a0a27c-8d2d-493b-ac0a-df36fb68699a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:03:49.669660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:49.669660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:49.669660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:49.669660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:49.669660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:49.669660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:49.669660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:49.669660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:49.669660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:49.669660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:49.669660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:49.669660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:03:49.669660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:03:49.669660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:49.670615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:49.671576 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:03:49.671576 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:03:49.671576 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:03:49.671576 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:03:49.671576 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 606/1210] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:49.677151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3013a4-be3b-4958-92dc-2ce0cb4b54f2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:49.774321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:49.777201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3013a4-be3b-4958-92dc-2ce0cb4b54f2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:49.811790 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa6a7a-3dbf-4f28-9ddd-21e6ad6ac065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:49.812451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa6a7a-3dbf-4f28-9ddd-21e6ad6ac065 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:49.812767 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa6a7a-3dbf-4f28-9ddd-21e6ad6ac065 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:49.813276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa6a7a-3dbf-4f28-9ddd-21e6ad6ac065 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:49.814125 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa6a7a-3dbf-4f28-9ddd-21e6ad6ac065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:49.814438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa6a7a-3dbf-4f28-9ddd-21e6ad6ac065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:49.815855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa6a7a-3dbf-4f28-9ddd-21e6ad6ac065 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:49.827016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 605/1211] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:49.833316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0a27c-8d2d-493b-ac0a-df36fb68699a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:49.932818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0a27c-8d2d-493b-ac0a-df36fb68699a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:49.934311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:49.967506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244a8cf-7fb2-4b32-9edc-3496aaaed629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:49.968306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244a8cf-7fb2-4b32-9edc-3496aaaed629 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:49.968587 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244a8cf-7fb2-4b32-9edc-3496aaaed629 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:49.968863 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244a8cf-7fb2-4b32-9edc-3496aaaed629 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:49.977252 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244a8cf-7fb2-4b32-9edc-3496aaaed629 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:49.978750 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244a8cf-7fb2-4b32-9edc-3496aaaed629 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-0244a8cf-7fb2-4b32-9edc-3496aaaed629 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:03:49.993568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:49.994861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:49.996317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:49.996317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:49.996317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:03:49.996317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:03:49.996317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:03:49.996317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:03:49.996317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:03:49.996317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:03:49.996317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 15:03:49.996317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:03:49.996317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 607/1212] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:03:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:50.001514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa6a7a-3dbf-4f28-9ddd-21e6ad6ac065 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:50.099797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa6a7a-3dbf-4f28-9ddd-21e6ad6ac065 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:50.132328 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be1814-bdf4-4ec1-9e58-daa722b5abe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:50.132989 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be1814-bdf4-4ec1-9e58-daa722b5abe3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:50.133267 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be1814-bdf4-4ec1-9e58-daa722b5abe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:50.133541 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be1814-bdf4-4ec1-9e58-daa722b5abe3 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:50.134355 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be1814-bdf4-4ec1-9e58-daa722b5abe3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:50.134659 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be1814-bdf4-4ec1-9e58-daa722b5abe3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:50.136092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be1814-bdf4-4ec1-9e58-daa722b5abe3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:50.153320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 606/1213] 10.222.0.67 () {56 vars in 1143 bytes} [Fri Jun 24 15:03:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:50.158888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244a8cf-7fb2-4b32-9edc-3496aaaed629 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:50.256035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244a8cf-7fb2-4b32-9edc-3496aaaed629 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:50.289118 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db0a2d-30d9-4fc1-97d8-2af350761e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:50.289760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db0a2d-30d9-4fc1-97d8-2af350761e55 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:50.290050 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db0a2d-30d9-4fc1-97d8-2af350761e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:50.290324 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db0a2d-30d9-4fc1-97d8-2af350761e55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:50.319641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:50.357784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db0a2d-30d9-4fc1-97d8-2af350761e55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:50.359271 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db0a2d-30d9-4fc1-97d8-2af350761e55 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:50.454414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 608/1214] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:03:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:03:50.460238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be1814-bdf4-4ec1-9e58-daa722b5abe3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:50.478497 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:50.556313 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be1814-bdf4-4ec1-9e58-daa722b5abe3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:50.590149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d055f37-6c42-45e5-acce-10d7fda840bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:50.590930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d055f37-6c42-45e5-acce-10d7fda840bd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:50.591236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d055f37-6c42-45e5-acce-10d7fda840bd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:50.591515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d055f37-6c42-45e5-acce-10d7fda840bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:50.601358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d055f37-6c42-45e5-acce-10d7fda840bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:50.602784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d055f37-6c42-45e5-acce-10d7fda840bd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:50.620555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-8d055f37-6c42-45e5-acce-10d7fda840bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:03:50.620555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:50.620555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:50.620555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:50.620555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:50.620555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:50.620555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:50.620555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:50.620555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:50.620555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:50.620555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:50.620555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:50.620555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:03:50.620555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:03:50.620555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:50.621655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:50.622961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:03:50.622961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:03:50.622961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:03:50.622961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:03:50.622961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 607/1215] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:50.628301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db0a2d-30d9-4fc1-97d8-2af350761e55 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:50.689890 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:50.726602 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db0a2d-30d9-4fc1-97d8-2af350761e55 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:50.759219 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8ab21-ccaf-4f98-a480-a650358ffc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:50.759884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8ab21-ccaf-4f98-a480-a650358ffc30 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:50.760165 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8ab21-ccaf-4f98-a480-a650358ffc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:50.760442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8ab21-ccaf-4f98-a480-a650358ffc30 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:50.761158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d8ab21-ccaf-4f98-a480-a650358ffc30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:50.761465 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d8ab21-ccaf-4f98-a480-a650358ffc30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:50.762817 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d8ab21-ccaf-4f98-a480-a650358ffc30 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:50.774044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 609/1216] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:50.779745 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d055f37-6c42-45e5-acce-10d7fda840bd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:50.849671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:50.879234 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d055f37-6c42-45e5-acce-10d7fda840bd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:50.912180 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01784c6a-d176-49ae-9645-844872769baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:50.913014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01784c6a-d176-49ae-9645-844872769baf None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:50.913306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01784c6a-d176-49ae-9645-844872769baf None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:50.913587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01784c6a-d176-49ae-9645-844872769baf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:50.920367 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01784c6a-d176-49ae-9645-844872769baf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:50.921809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01784c6a-d176-49ae-9645-844872769baf None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:50.936473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-01784c6a-d176-49ae-9645-844872769baf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:03:50.936473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:50.936473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:50.936473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:50.936473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:50.936473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:50.936473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:50.936473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:50.936473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:50.936473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:50.936473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:50.936473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:50.936473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:03:50.936473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:03:50.936473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:50.937547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:50.938651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:03:50.938651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:03:50.938651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:03:50.938651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:03:50.938651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:03:50.938651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:03:50.938651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:03:50.938651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:03:50.938651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 608/1217] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:03:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:50.943667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8ab21-ccaf-4f98-a480-a650358ffc30 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:51.040953 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8ab21-ccaf-4f98-a480-a650358ffc30 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:51.063083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:51.074288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9724de4-d71c-4a32-8f7e-0b13e8dfff35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:51.074937 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9724de4-d71c-4a32-8f7e-0b13e8dfff35 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:51.075269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9724de4-d71c-4a32-8f7e-0b13e8dfff35 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:51.075543 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9724de4-d71c-4a32-8f7e-0b13e8dfff35 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:51.076321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9724de4-d71c-4a32-8f7e-0b13e8dfff35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:51.076626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9724de4-d71c-4a32-8f7e-0b13e8dfff35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:51.078086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9724de4-d71c-4a32-8f7e-0b13e8dfff35 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:51.092832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 610/1218] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:03:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:51.096430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01784c6a-d176-49ae-9645-844872769baf None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:51.197933 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01784c6a-d176-49ae-9645-844872769baf None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:51.225685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:51.231287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89275dd0-095f-4fa3-8407-42ed71f9ae59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:51.232012 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89275dd0-095f-4fa3-8407-42ed71f9ae59 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:51.232306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89275dd0-095f-4fa3-8407-42ed71f9ae59 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:51.232584 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89275dd0-095f-4fa3-8407-42ed71f9ae59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:51.233413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89275dd0-095f-4fa3-8407-42ed71f9ae59 None admin] RBAC: Adding query filter params (role.id=716cc96fb28146b7b9ae6271569a206f, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=f612d6ff06fc458e8c1178cbf8321f79) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:51.233730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89275dd0-095f-4fa3-8407-42ed71f9ae59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:51.236071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89275dd0-095f-4fa3-8407-42ed71f9ae59 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:51.241472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 609/1219] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:03:51 2022] GET /identity/v3/role_assignments?user.id=f612d6ff06fc458e8c1178cbf8321f79&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=716cc96fb28146b7b9ae6271569a206f => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:53.247711 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5a440-0b17-47f1-b876-fce19f332496 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:53.248262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5a440-0b17-47f1-b876-fce19f332496 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:53.248360 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5a440-0b17-47f1-b876-fce19f332496 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:53.249761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 611/1220] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:03:53.254938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ba985-281b-4a2e-8fa3-ed0338e5bc8c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:53.255357 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ba985-281b-4a2e-8fa3-ed0338e5bc8c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:53.255538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ba985-281b-4a2e-8fa3-ed0338e5bc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:53.272037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0e4ba985-281b-4a2e-8fa3-ed0338e5bc8c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:03:53.273193 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0e4ba985-281b-4a2e-8fa3-ed0338e5bc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:03:53.340144 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 610/1221] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:03:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:03:53.364738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5a440-0b17-47f1-b876-fce19f332496 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:53.464890 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5a440-0b17-47f1-b876-fce19f332496 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:53.498222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9631067-6dbc-425e-83fb-c4acd8c46b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:53.499007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9631067-6dbc-425e-83fb-c4acd8c46b48 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:53.499325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9631067-6dbc-425e-83fb-c4acd8c46b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:53.499600 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9631067-6dbc-425e-83fb-c4acd8c46b48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:53.509468 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9631067-6dbc-425e-83fb-c4acd8c46b48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a9631067-6dbc-425e-83fb-c4acd8c46b48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:03:53.517676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:53.518639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:53.519525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:53.519525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:53.519525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:53.519525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:53.519525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:03:53.519525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:03:53.519525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:03:53.519525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:03:53.519525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:03:53.519525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:03:53.519525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:03:53.519525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:03:53.519525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 612/1222] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:03:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:53.525037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ba985-281b-4a2e-8fa3-ed0338e5bc8c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:53.621521 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ba985-281b-4a2e-8fa3-ed0338e5bc8c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:53.644428 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:53.654736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7e794-989c-4a18-a2b6-e7b26dd4bf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:53.655429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7e794-989c-4a18-a2b6-e7b26dd4bf23 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:53.655714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7e794-989c-4a18-a2b6-e7b26dd4bf23 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:53.655990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7e794-989c-4a18-a2b6-e7b26dd4bf23 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:53.657481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7e794-989c-4a18-a2b6-e7b26dd4bf23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:53.657804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7e794-989c-4a18-a2b6-e7b26dd4bf23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:53.659163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7e794-989c-4a18-a2b6-e7b26dd4bf23 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:53.666852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 611/1223] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:03:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:53.673544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9631067-6dbc-425e-83fb-c4acd8c46b48 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:53.717268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:53.772225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9631067-6dbc-425e-83fb-c4acd8c46b48 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:53.806839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42c5f2-cee1-4b26-9b51-8c3079661ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:53.807728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42c5f2-cee1-4b26-9b51-8c3079661ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:53.808015 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42c5f2-cee1-4b26-9b51-8c3079661ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:53.808295 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42c5f2-cee1-4b26-9b51-8c3079661ca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:53.818677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc42c5f2-cee1-4b26-9b51-8c3079661ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:53.820249 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc42c5f2-cee1-4b26-9b51-8c3079661ca3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:53.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-bc42c5f2-cee1-4b26-9b51-8c3079661ca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:03:53.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:53.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:53.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:53.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:53.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:53.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:53.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:53.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:53.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:53.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:53.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:53.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:03:53.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:03:53.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:53.839352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:53.840543 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:03:53.840543 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:03:53.840543 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:03:53.840543 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:03:53.840543 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 613/1224] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:53.845540 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7e794-989c-4a18-a2b6-e7b26dd4bf23 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:53.943606 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7e794-989c-4a18-a2b6-e7b26dd4bf23 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:53.976401 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb989d-7b25-47eb-ad37-b0a4f1a66a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:53.977074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb989d-7b25-47eb-ad37-b0a4f1a66a60 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:53.977357 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb989d-7b25-47eb-ad37-b0a4f1a66a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:53.977633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb989d-7b25-47eb-ad37-b0a4f1a66a60 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:53.978346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb989d-7b25-47eb-ad37-b0a4f1a66a60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:53.978651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb989d-7b25-47eb-ad37-b0a4f1a66a60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:53.980037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb989d-7b25-47eb-ad37-b0a4f1a66a60 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:53.991546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 612/1225] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:53.997272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42c5f2-cee1-4b26-9b51-8c3079661ca3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:54.088936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:54.097022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42c5f2-cee1-4b26-9b51-8c3079661ca3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:54.129733 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22926590-42a4-44da-9f3b-03f210dd0e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:54.130582 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22926590-42a4-44da-9f3b-03f210dd0e93 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:54.130861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22926590-42a4-44da-9f3b-03f210dd0e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:54.131132 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22926590-42a4-44da-9f3b-03f210dd0e93 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:54.137858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22926590-42a4-44da-9f3b-03f210dd0e93 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:54.139317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22926590-42a4-44da-9f3b-03f210dd0e93 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-22926590-42a4-44da-9f3b-03f210dd0e93 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:03:54.154187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:54.158125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:54.159352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:54.159352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:54.159352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:03:54.159352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:03:54.159352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:03:54.159352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:03:54.159352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:03:54.159352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:03:54.159352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 15:03:54.159352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:03:54.159352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 614/1226] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:03:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:54.161345 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb989d-7b25-47eb-ad37-b0a4f1a66a60 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:54.209027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:54.263045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb989d-7b25-47eb-ad37-b0a4f1a66a60 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:54.296649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55920e-96a2-4be6-9027-e94d4667c494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:54.298093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55920e-96a2-4be6-9027-e94d4667c494 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:54.298379 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55920e-96a2-4be6-9027-e94d4667c494 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:54.298654 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55920e-96a2-4be6-9027-e94d4667c494 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:54.299515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55920e-96a2-4be6-9027-e94d4667c494 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:54.299824 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55920e-96a2-4be6-9027-e94d4667c494 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:54.301244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55920e-96a2-4be6-9027-e94d4667c494 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:54.319738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 613/1227] 10.222.0.67 () {56 vars in 1143 bytes} [Fri Jun 24 15:03:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:54.325494 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22926590-42a4-44da-9f3b-03f210dd0e93 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:54.423420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22926590-42a4-44da-9f3b-03f210dd0e93 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:54.456417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea72ed-8f96-49cd-a69d-8d7e4fdd7531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:54.457243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea72ed-8f96-49cd-a69d-8d7e4fdd7531 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:54.457531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea72ed-8f96-49cd-a69d-8d7e4fdd7531 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:54.457814 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea72ed-8f96-49cd-a69d-8d7e4fdd7531 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:54.467760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea72ed-8f96-49cd-a69d-8d7e4fdd7531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:54.469309 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea72ed-8f96-49cd-a69d-8d7e4fdd7531 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:54.487090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-6fea72ed-8f96-49cd-a69d-8d7e4fdd7531 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:03:54.487090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:54.487090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:54.487090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:54.487090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:54.487090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:54.487090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:54.487090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:54.487090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:54.487090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:54.487090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:54.487090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:54.487090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:03:54.487090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:03:54.487090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:54.488175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:54.489140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:03:54.489140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:03:54.489140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:03:54.489140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:03:54.489140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 615/1228] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:54.494181 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55920e-96a2-4be6-9027-e94d4667c494 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:54.592576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55920e-96a2-4be6-9027-e94d4667c494 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:54.595408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:54.625931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935945c-cf23-43ae-ae84-baf77ffae8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:54.626613 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935945c-cf23-43ae-ae84-baf77ffae8ff None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:54.626893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935945c-cf23-43ae-ae84-baf77ffae8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:54.627168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935945c-cf23-43ae-ae84-baf77ffae8ff None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:54.627901 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935945c-cf23-43ae-ae84-baf77ffae8ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:54.628204 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935945c-cf23-43ae-ae84-baf77ffae8ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:54.629545 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935945c-cf23-43ae-ae84-baf77ffae8ff None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:54.640289 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 614/1229] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:54.646064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea72ed-8f96-49cd-a69d-8d7e4fdd7531 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:54.656882 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:54.742622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea72ed-8f96-49cd-a69d-8d7e4fdd7531 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:54.774485 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae1386-67c0-4112-95e8-091484ec76d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:54.775306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae1386-67c0-4112-95e8-091484ec76d1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:54.775593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae1386-67c0-4112-95e8-091484ec76d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:54.775872 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae1386-67c0-4112-95e8-091484ec76d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:54.783335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae1386-67c0-4112-95e8-091484ec76d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:54.788652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae1386-67c0-4112-95e8-091484ec76d1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:54.800605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-03ae1386-67c0-4112-95e8-091484ec76d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:03:54.800605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:54.800605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:54.800605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:54.800605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:54.800605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:54.800605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:54.800605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:54.800605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:54.800605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:54.800605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:54.800605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:54.800605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:03:54.800605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:03:54.800605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:54.801652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:54.802875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:03:54.802875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:03:54.802875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:03:54.802875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:03:54.802875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:03:54.802875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:03:54.802875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:03:54.802875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:03:54.802875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 616/1230] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:03:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:54.807698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935945c-cf23-43ae-ae84-baf77ffae8ff None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:54.906293 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935945c-cf23-43ae-ae84-baf77ffae8ff None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:54.939563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94900ebe-11da-4798-a689-86f030ac5039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:54.940214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94900ebe-11da-4798-a689-86f030ac5039 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:54.940492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94900ebe-11da-4798-a689-86f030ac5039 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:54.940767 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94900ebe-11da-4798-a689-86f030ac5039 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:54.941592 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94900ebe-11da-4798-a689-86f030ac5039 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:54.941895 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94900ebe-11da-4798-a689-86f030ac5039 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:54.943323 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94900ebe-11da-4798-a689-86f030ac5039 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:54.955699 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 615/1231] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:03:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:54.961672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae1386-67c0-4112-95e8-091484ec76d1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:55.028065 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:55.064186 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae1386-67c0-4112-95e8-091484ec76d1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:55.099446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145903ba-1f83-40a4-8f37-f8b492b7c173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:55.100346 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145903ba-1f83-40a4-8f37-f8b492b7c173 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:55.100635 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145903ba-1f83-40a4-8f37-f8b492b7c173 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:55.100915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145903ba-1f83-40a4-8f37-f8b492b7c173 None admin] PATH_INFO: `/v3/projects/06da054e8b7545dfa56780ab56585be8/users/f612d6ff06fc458e8c1178cbf8321f79/roles/716cc96fb28146b7b9ae6271569a206f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:55.114828 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145903ba-1f83-40a4-8f37-f8b492b7c173 None admin] RBAC: Authorizing `identity:create_grant(project_id=06da054e8b7545dfa56780ab56585be8, user_id=f612d6ff06fc458e8c1178cbf8321f79, role_id=716cc96fb28146b7b9ae6271569a206f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:55.116288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145903ba-1f83-40a4-8f37-f8b492b7c173 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:55.200098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 617/1232] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:03:54 2022] PUT /identity/v3/projects/06da054e8b7545dfa56780ab56585be8/users/f612d6ff06fc458e8c1178cbf8321f79/roles/716cc96fb28146b7b9ae6271569a206f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:03:57.243581 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbe97b-1afb-4506-b8c7-f232649b0f8f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:57.244181 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbe97b-1afb-4506-b8c7-f232649b0f8f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:57.244181 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbe97b-1afb-4506-b8c7-f232649b0f8f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:57.245594 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 616/1233] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:03:57.251319 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cad19d-2977-41fd-a424-afcf7c677362 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:57.253785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cad19d-2977-41fd-a424-afcf7c677362 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:57.253785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cad19d-2977-41fd-a424-afcf7c677362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:57.267947 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-87cad19d-2977-41fd-a424-afcf7c677362 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:03:57.269128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-87cad19d-2977-41fd-a424-afcf7c677362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:03:57.335740 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 618/1234] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:03:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:03:57.360451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbe97b-1afb-4506-b8c7-f232649b0f8f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:57.385923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:57.460695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbe97b-1afb-4506-b8c7-f232649b0f8f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:57.494706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a2f98-a738-47dc-88c8-a32687979448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:57.495521 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a2f98-a738-47dc-88c8-a32687979448 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:57.495831 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a2f98-a738-47dc-88c8-a32687979448 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:57.496106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a2f98-a738-47dc-88c8-a32687979448 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:57.507118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a2f98-a738-47dc-88c8-a32687979448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d22a2f98-a738-47dc-88c8-a32687979448 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:03:57.514871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:57.516151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:57.517303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:57.517303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:57.517303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:57.517303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:57.517303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:03:57.517303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:03:57.517303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:03:57.517303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:03:57.517303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:03:57.517303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:03:57.517303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 15:03:57.517303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:03:57.517303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 617/1235] 10.222.0.67 () {56 vars in 1102 bytes} [Fri Jun 24 15:03:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:57.522375 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cad19d-2977-41fd-a424-afcf7c677362 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:57.568733 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:57.620683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cad19d-2977-41fd-a424-afcf7c677362 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:57.652963 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e94c69-d8a1-4747-896e-af5591b6d225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:57.658690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e94c69-d8a1-4747-896e-af5591b6d225 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:57.658690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e94c69-d8a1-4747-896e-af5591b6d225 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:57.658690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e94c69-d8a1-4747-896e-af5591b6d225 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:57.658690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e94c69-d8a1-4747-896e-af5591b6d225 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:57.658690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e94c69-d8a1-4747-896e-af5591b6d225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:57.658690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e94c69-d8a1-4747-896e-af5591b6d225 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:57.665507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 619/1236] 10.222.0.67 () {56 vars in 1103 bytes} [Fri Jun 24 15:03:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:57.671142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a2f98-a738-47dc-88c8-a32687979448 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:57.757347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:57.772732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a2f98-a738-47dc-88c8-a32687979448 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:57.806436 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14588979-217f-401e-bd90-ef7be69f07f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:57.807258 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14588979-217f-401e-bd90-ef7be69f07f7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:57.807548 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14588979-217f-401e-bd90-ef7be69f07f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:57.807823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14588979-217f-401e-bd90-ef7be69f07f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:57.817612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14588979-217f-401e-bd90-ef7be69f07f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:57.819349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14588979-217f-401e-bd90-ef7be69f07f7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:57.836072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-14588979-217f-401e-bd90-ef7be69f07f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:03:57.836072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:57.836072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:57.836072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:57.836072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:57.836072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:57.836072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:57.836072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:57.836072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:57.836072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:57.836072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:57.836072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:57.836072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:03:57.836072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:03:57.836072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:57.837207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:57.838439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:03:57.838439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:03:57.838439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:03:57.838439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:03:57.838439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 618/1237] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:03:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:57.843157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e94c69-d8a1-4747-896e-af5591b6d225 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:57.943534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e94c69-d8a1-4747-896e-af5591b6d225 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:57.952260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:57.977374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7511258-a6d2-48e5-b568-e8a4b6a987e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:57.978033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7511258-a6d2-48e5-b568-e8a4b6a987e9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:57.978318 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7511258-a6d2-48e5-b568-e8a4b6a987e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:57.978596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7511258-a6d2-48e5-b568-e8a4b6a987e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:57.979344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7511258-a6d2-48e5-b568-e8a4b6a987e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:57.979652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7511258-a6d2-48e5-b568-e8a4b6a987e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:57.981010 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7511258-a6d2-48e5-b568-e8a4b6a987e9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:57.992498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 620/1238] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:57.998183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14588979-217f-401e-bd90-ef7be69f07f7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:58.097667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14588979-217f-401e-bd90-ef7be69f07f7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:58.130537 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71409b80-bba7-460e-88eb-49d4d4a46b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:58.131365 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71409b80-bba7-460e-88eb-49d4d4a46b54 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:58.131695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71409b80-bba7-460e-88eb-49d4d4a46b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:58.131974 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71409b80-bba7-460e-88eb-49d4d4a46b54 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:58.134623 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:58.138986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71409b80-bba7-460e-88eb-49d4d4a46b54 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:58.140957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71409b80-bba7-460e-88eb-49d4d4a46b54 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-71409b80-bba7-460e-88eb-49d4d4a46b54 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:03:58.155339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:58.157060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:58.158341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:58.158341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:58.158341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:03:58.158341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:03:58.158341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:03:58.158341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:03:58.158341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:03:58.158341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:03:58.158341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 15:03:58.158341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:03:58.158341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 619/1239] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:03:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:58.162775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7511258-a6d2-48e5-b568-e8a4b6a987e9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:58.259484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7511258-a6d2-48e5-b568-e8a4b6a987e9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:58.292635 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2076d-5575-428d-b0ba-ae313b2cd71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:58.296386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2076d-5575-428d-b0ba-ae313b2cd71d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:58.296386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2076d-5575-428d-b0ba-ae313b2cd71d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:58.296386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2076d-5575-428d-b0ba-ae313b2cd71d None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:58.296386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca2076d-5575-428d-b0ba-ae313b2cd71d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:58.296386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca2076d-5575-428d-b0ba-ae313b2cd71d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:58.296386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca2076d-5575-428d-b0ba-ae313b2cd71d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:58.314458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 621/1240] 10.222.0.67 () {56 vars in 1143 bytes} [Fri Jun 24 15:03:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:58.320162 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71409b80-bba7-460e-88eb-49d4d4a46b54 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:58.418494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71409b80-bba7-460e-88eb-49d4d4a46b54 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:58.452249 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a99df8-4fd3-4725-b6a5-eac896833ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:58.452888 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a99df8-4fd3-4725-b6a5-eac896833ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:58.454014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a99df8-4fd3-4725-b6a5-eac896833ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:58.454288 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a99df8-4fd3-4725-b6a5-eac896833ce9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:58.512452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:58.519412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a99df8-4fd3-4725-b6a5-eac896833ce9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:58.520854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a99df8-4fd3-4725-b6a5-eac896833ce9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:58.615870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 620/1241] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:03:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:03:58.621755 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2076d-5575-428d-b0ba-ae313b2cd71d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:58.649903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:58.721730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2076d-5575-428d-b0ba-ae313b2cd71d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:58.755260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39251e-c151-4638-bbf9-2b413fc069b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:58.756057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39251e-c151-4638-bbf9-2b413fc069b1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:58.756340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39251e-c151-4638-bbf9-2b413fc069b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:58.756615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39251e-c151-4638-bbf9-2b413fc069b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:58.766522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39251e-c151-4638-bbf9-2b413fc069b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:58.767926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39251e-c151-4638-bbf9-2b413fc069b1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:58.785230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-1c39251e-c151-4638-bbf9-2b413fc069b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:03:58.785230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:58.785230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:58.785230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:58.785230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:58.785230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:58.785230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:58.785230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:58.785230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:58.785230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:58.785230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:58.785230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:58.785230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:03:58.785230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:03:58.785230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:58.786207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:58.787313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:03:58.787313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:03:58.787313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:03:58.787313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:03:58.787313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 622/1242] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:03:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:58.792312 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a99df8-4fd3-4725-b6a5-eac896833ce9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:58.883244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:58.888925 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a99df8-4fd3-4725-b6a5-eac896833ce9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:58.922546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d5217-6470-4fa4-941c-ec815f1fef4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:58.923500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d5217-6470-4fa4-941c-ec815f1fef4c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:58.923780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d5217-6470-4fa4-941c-ec815f1fef4c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:58.924053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d5217-6470-4fa4-941c-ec815f1fef4c None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:58.924759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d5217-6470-4fa4-941c-ec815f1fef4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:58.925058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d5217-6470-4fa4-941c-ec815f1fef4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:58.926395 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d5217-6470-4fa4-941c-ec815f1fef4c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:58.937141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 621/1243] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:58.943680 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39251e-c151-4638-bbf9-2b413fc069b1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:59.022033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:03:59.045561 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39251e-c151-4638-bbf9-2b413fc069b1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:59.080287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17211d91-5f7e-4fa7-92e3-97c84d8dde01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:59.081085 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17211d91-5f7e-4fa7-92e3-97c84d8dde01 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:59.081371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17211d91-5f7e-4fa7-92e3-97c84d8dde01 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:59.081651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17211d91-5f7e-4fa7-92e3-97c84d8dde01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:59.088688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17211d91-5f7e-4fa7-92e3-97c84d8dde01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:59.092385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17211d91-5f7e-4fa7-92e3-97c84d8dde01 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:59.106572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-17211d91-5f7e-4fa7-92e3-97c84d8dde01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:03:59.106572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:03:59.106572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:03:59.106572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:03:59.106572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:03:59.106572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:03:59.106572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:03:59.106572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:03:59.106572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:03:59.106572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:03:59.106572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:03:59.106572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:03:59.106572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:03:59.106572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:03:59.106572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:03:59.107796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:03:59.108957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:03:59.108957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:03:59.108957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:03:59.108957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:03:59.108957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:03:59.108957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:03:59.108957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:03:59.108957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:03:59.108957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 623/1244] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:03:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:03:59.114142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d5217-6470-4fa4-941c-ec815f1fef4c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:59.214665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d5217-6470-4fa4-941c-ec815f1fef4c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:59.251293 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf00d1e-bd30-47da-a03d-ef5901364efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:59.251961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf00d1e-bd30-47da-a03d-ef5901364efa None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:59.252280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf00d1e-bd30-47da-a03d-ef5901364efa None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:59.252594 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf00d1e-bd30-47da-a03d-ef5901364efa None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:59.253475 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf00d1e-bd30-47da-a03d-ef5901364efa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:59.253794 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf00d1e-bd30-47da-a03d-ef5901364efa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:59.255256 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf00d1e-bd30-47da-a03d-ef5901364efa None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:59.267989 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 622/1245] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:03:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:03:59.273478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17211d91-5f7e-4fa7-92e3-97c84d8dde01 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:03:59.373477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17211d91-5f7e-4fa7-92e3-97c84d8dde01 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:03:59.407939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2cc5c-ad6b-412f-b319-a193ed25ffad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:03:59.408619 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2cc5c-ad6b-412f-b319-a193ed25ffad None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:03:59.408908 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2cc5c-ad6b-412f-b319-a193ed25ffad None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:03:59.409198 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2cc5c-ad6b-412f-b319-a193ed25ffad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:03:59.410025 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2cc5c-ad6b-412f-b319-a193ed25ffad None admin] RBAC: Adding query filter params (role.id=716cc96fb28146b7b9ae6271569a206f, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=f612d6ff06fc458e8c1178cbf8321f79) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:03:59.410343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2cc5c-ad6b-412f-b319-a193ed25ffad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:03:59.411813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2cc5c-ad6b-412f-b319-a193ed25ffad None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:03:59.417649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 624/1246] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:03:59 2022] GET /identity/v3/role_assignments?user.id=f612d6ff06fc458e8c1178cbf8321f79&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=716cc96fb28146b7b9ae6271569a206f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:01.491504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ba6bd-4134-4725-b5dc-9a1913487dc2 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:01.492063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ba6bd-4134-4725-b5dc-9a1913487dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:01.492063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ba6bd-4134-4725-b5dc-9a1913487dc2 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:01.493423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 623/1247] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:04:01.498532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074981e-94fd-4e16-8ad2-51bee663f9f5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:01.498976 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074981e-94fd-4e16-8ad2-51bee663f9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:01.499095 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074981e-94fd-4e16-8ad2-51bee663f9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:01.504815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:01.517511 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2074981e-94fd-4e16-8ad2-51bee663f9f5 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:04:01.518684 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2074981e-94fd-4e16-8ad2-51bee663f9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:04:01.585922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 625/1248] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:04:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:04:01.609753 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ba6bd-4134-4725-b5dc-9a1913487dc2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:01.628760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:01.710421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ba6bd-4134-4725-b5dc-9a1913487dc2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:01.745992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e9df0-d187-4658-a59c-6c2b07af6081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:01.746720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e9df0-d187-4658-a59c-6c2b07af6081 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:01.747221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e9df0-d187-4658-a59c-6c2b07af6081 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:01.747368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e9df0-d187-4658-a59c-6c2b07af6081 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:01.758628 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e9df0-d187-4658-a59c-6c2b07af6081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-7e2e9df0-d187-4658-a59c-6c2b07af6081 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:04:01.766772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:01.767960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:01.769038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:01.769038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:01.769038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:01.769038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:01.769038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:04:01.769038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:04:01.769038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:04:01.769038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:04:01.769038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:04:01.769038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:04:01.769038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:04:01.769038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:04:01.769038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 624/1249] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:04:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:01.774156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074981e-94fd-4e16-8ad2-51bee663f9f5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:01.873645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074981e-94fd-4e16-8ad2-51bee663f9f5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:01.902660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:01.907721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77bf46-ef26-4df5-afc7-2781cd8782f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:01.908409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77bf46-ef26-4df5-afc7-2781cd8782f6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:01.908689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77bf46-ef26-4df5-afc7-2781cd8782f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:01.908963 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77bf46-ef26-4df5-afc7-2781cd8782f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:01.909702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77bf46-ef26-4df5-afc7-2781cd8782f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:01.910006 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77bf46-ef26-4df5-afc7-2781cd8782f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:01.911386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77bf46-ef26-4df5-afc7-2781cd8782f6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:01.919135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 626/1250] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:04:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:01.925854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e9df0-d187-4658-a59c-6c2b07af6081 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:02.003486 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:02.024982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e9df0-d187-4658-a59c-6c2b07af6081 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:02.058938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2c1b8-201f-42bc-ae43-5465331fa9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:02.059784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2c1b8-201f-42bc-ae43-5465331fa9da None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:02.060068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2c1b8-201f-42bc-ae43-5465331fa9da None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:02.060344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2c1b8-201f-42bc-ae43-5465331fa9da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:02.070638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2c1b8-201f-42bc-ae43-5465331fa9da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:02.072207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2c1b8-201f-42bc-ae43-5465331fa9da None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:02.089402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-efc2c1b8-201f-42bc-ae43-5465331fa9da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:04:02.089402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:02.089402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:02.089402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:02.089402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:02.089402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:02.089402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:02.089402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:02.089402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:02.089402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:02.089402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:02.089402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:02.089402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:04:02.089402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:04:02.089402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:02.090515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:02.096746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:04:02.096746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:04:02.096746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:04:02.096746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:04:02.096746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 625/1251] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:02.097616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77bf46-ef26-4df5-afc7-2781cd8782f6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:02.196061 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77bf46-ef26-4df5-afc7-2781cd8782f6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:02.228536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f76df-ff17-403c-8c6d-9748a24c2350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:02.229210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f76df-ff17-403c-8c6d-9748a24c2350 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:02.229492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f76df-ff17-403c-8c6d-9748a24c2350 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:02.229767 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f76df-ff17-403c-8c6d-9748a24c2350 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:02.230486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f76df-ff17-403c-8c6d-9748a24c2350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:02.230837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f76df-ff17-403c-8c6d-9748a24c2350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:02.232248 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f76df-ff17-403c-8c6d-9748a24c2350 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:02.244595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 627/1252] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:02.250343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2c1b8-201f-42bc-ae43-5465331fa9da None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:02.347477 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2c1b8-201f-42bc-ae43-5465331fa9da None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:02.382044 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:02.390087 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465e247-e46f-4a4d-93aa-8a0081e2ff7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:02.391438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465e247-e46f-4a4d-93aa-8a0081e2ff7c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:02.391720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465e247-e46f-4a4d-93aa-8a0081e2ff7c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:02.391991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465e247-e46f-4a4d-93aa-8a0081e2ff7c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:02.399825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465e247-e46f-4a4d-93aa-8a0081e2ff7c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:02.401297 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465e247-e46f-4a4d-93aa-8a0081e2ff7c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-e465e247-e46f-4a4d-93aa-8a0081e2ff7c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:04:02.423806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:02.424949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:02.425921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:02.425921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:02.425921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:04:02.425921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:04:02.425921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:04:02.425921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:04:02.425921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:04:02.425921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:04:02.425921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 15:04:02.425921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:04:02.425921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 626/1253] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:04:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:02.431600 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f76df-ff17-403c-8c6d-9748a24c2350 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:02.512712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:02.558222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f76df-ff17-403c-8c6d-9748a24c2350 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:02.590721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e111de-fff1-45da-8abf-b24e63ced7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:02.591420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e111de-fff1-45da-8abf-b24e63ced7cf None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:02.591701 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e111de-fff1-45da-8abf-b24e63ced7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:02.591974 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e111de-fff1-45da-8abf-b24e63ced7cf None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:02.592806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e111de-fff1-45da-8abf-b24e63ced7cf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:02.593186 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e111de-fff1-45da-8abf-b24e63ced7cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:02.595344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e111de-fff1-45da-8abf-b24e63ced7cf None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:02.615774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 628/1254] 10.222.0.67 () {56 vars in 1143 bytes} [Fri Jun 24 15:04:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:02.617537 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465e247-e46f-4a4d-93aa-8a0081e2ff7c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:02.718644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465e247-e46f-4a4d-93aa-8a0081e2ff7c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:02.751788 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ae589-808c-4daf-9c59-15df26fd965e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:02.752471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ae589-808c-4daf-9c59-15df26fd965e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:02.752755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ae589-808c-4daf-9c59-15df26fd965e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:02.753033 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ae589-808c-4daf-9c59-15df26fd965e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:02.807427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:02.819636 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ae589-808c-4daf-9c59-15df26fd965e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:02.821078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ae589-808c-4daf-9c59-15df26fd965e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:02.918225 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 627/1255] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:04:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:04:02.925016 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e111de-fff1-45da-8abf-b24e63ced7cf None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:03.023092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e111de-fff1-45da-8abf-b24e63ced7cf None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:03.036214 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:03.057458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35cb0a6-6798-4f87-aa7f-eb7c5105b0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:03.058242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35cb0a6-6798-4f87-aa7f-eb7c5105b0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:03.058522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35cb0a6-6798-4f87-aa7f-eb7c5105b0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:03.058795 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35cb0a6-6798-4f87-aa7f-eb7c5105b0f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:03.068654 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35cb0a6-6798-4f87-aa7f-eb7c5105b0f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:03.070082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35cb0a6-6798-4f87-aa7f-eb7c5105b0f9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:03.087754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f35cb0a6-6798-4f87-aa7f-eb7c5105b0f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:04:03.087754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:03.087754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:03.087754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:03.087754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:03.087754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:03.087754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:03.087754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:03.087754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:03.087754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:03.087754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:03.087754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:03.087754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:04:03.087754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:04:03.087754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:03.088832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:03.089784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:04:03.089784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:04:03.089784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:04:03.089784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:04:03.089784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 629/1256] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:04:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:03.095610 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ae589-808c-4daf-9c59-15df26fd965e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:03.182741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:03.194502 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ae589-808c-4daf-9c59-15df26fd965e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:03.228594 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541813b-6915-441e-8774-390e8a870501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:03.229256 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541813b-6915-441e-8774-390e8a870501 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:03.229539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541813b-6915-441e-8774-390e8a870501 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:03.229813 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541813b-6915-441e-8774-390e8a870501 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:03.230527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6541813b-6915-441e-8774-390e8a870501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:03.230833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6541813b-6915-441e-8774-390e8a870501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:03.232218 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6541813b-6915-441e-8774-390e8a870501 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:03.243109 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 628/1257] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:03.248986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35cb0a6-6798-4f87-aa7f-eb7c5105b0f9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:03.351574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35cb0a6-6798-4f87-aa7f-eb7c5105b0f9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:03.384709 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0093b2c-cfa5-4a51-b2db-75a20971d060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:03.385500 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0093b2c-cfa5-4a51-b2db-75a20971d060 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:03.385785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0093b2c-cfa5-4a51-b2db-75a20971d060 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:03.386064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0093b2c-cfa5-4a51-b2db-75a20971d060 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:03.393535 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0093b2c-cfa5-4a51-b2db-75a20971d060 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:03.395025 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0093b2c-cfa5-4a51-b2db-75a20971d060 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:03.409257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f0093b2c-cfa5-4a51-b2db-75a20971d060 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:04:03.409257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:03.409257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:03.409257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:03.409257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:03.409257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:03.409257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:03.409257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:03.409257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:03.409257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:03.409257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:03.409257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:03.409257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:04:03.409257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:04:03.409257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:03.410272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:03.411333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:04:03.411333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:04:03.411333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:04:03.411333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:04:03.411333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:04:03.411333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:04:03.411333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:04:03.411333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:04:03.411333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 630/1258] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:04:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:03.416383 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541813b-6915-441e-8774-390e8a870501 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:03.517740 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541813b-6915-441e-8774-390e8a870501 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:03.551080 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adef67-0f1d-42ce-835a-18d471df3d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:03.551760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adef67-0f1d-42ce-835a-18d471df3d9a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:03.552040 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adef67-0f1d-42ce-835a-18d471df3d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:03.552332 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adef67-0f1d-42ce-835a-18d471df3d9a None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:03.553103 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36adef67-0f1d-42ce-835a-18d471df3d9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:03.553406 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36adef67-0f1d-42ce-835a-18d471df3d9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:03.554830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36adef67-0f1d-42ce-835a-18d471df3d9a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:03.566907 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 629/1259] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:04:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:03.573138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0093b2c-cfa5-4a51-b2db-75a20971d060 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:03.600946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:03.674805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0093b2c-cfa5-4a51-b2db-75a20971d060 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:03.709004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900beb6-6944-4b3b-8455-f16236622ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:03.709652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900beb6-6944-4b3b-8455-f16236622ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:03.709931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900beb6-6944-4b3b-8455-f16236622ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:03.710201 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900beb6-6944-4b3b-8455-f16236622ec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:03.711016 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900beb6-6944-4b3b-8455-f16236622ec6 None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=f612d6ff06fc458e8c1178cbf8321f79) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:03.711355 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900beb6-6944-4b3b-8455-f16236622ec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:03.712762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900beb6-6944-4b3b-8455-f16236622ec6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:03.717980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 631/1260] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:04:03 2022] GET /identity/v3/role_assignments?user.id=f612d6ff06fc458e8c1178cbf8321f79&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:05.726775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658e368-d80f-4d4e-8414-726878f1e0fe None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:05.727328 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658e368-d80f-4d4e-8414-726878f1e0fe None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:05.727446 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658e368-d80f-4d4e-8414-726878f1e0fe None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:05.728738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 630/1261] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:04:05.734586 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e004b-10e2-416a-bb12-5aa5b65cdb52 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:05.735018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e004b-10e2-416a-bb12-5aa5b65cdb52 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:05.735470 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e004b-10e2-416a-bb12-5aa5b65cdb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:05.752591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f19e004b-10e2-416a-bb12-5aa5b65cdb52 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:04:05.753782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f19e004b-10e2-416a-bb12-5aa5b65cdb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:04:05.821680 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 632/1262] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:04:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:04:05.846045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658e368-d80f-4d4e-8414-726878f1e0fe None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:05.856815 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:05.946510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658e368-d80f-4d4e-8414-726878f1e0fe None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:05.980418 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd229428-4cd3-4504-884a-dffe05f4453f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:05.981151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd229428-4cd3-4504-884a-dffe05f4453f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:05.981437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd229428-4cd3-4504-884a-dffe05f4453f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:05.981715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd229428-4cd3-4504-884a-dffe05f4453f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:05.992500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd229428-4cd3-4504-884a-dffe05f4453f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-bd229428-4cd3-4504-884a-dffe05f4453f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:04:06.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:06.002190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:06.003368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:06.003368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:06.003368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:06.003368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:06.003368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:04:06.003368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:04:06.003368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:04:06.003368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:04:06.003368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:04:06.003368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:04:06.003368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:04:06.003368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:04:06.003368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 631/1263] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:04:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:06.009433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e004b-10e2-416a-bb12-5aa5b65cdb52 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:06.028038 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:06.110282 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e004b-10e2-416a-bb12-5aa5b65cdb52 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:06.143027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c79641-49de-4af9-bf0c-f4a2225d2bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:06.143727 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c79641-49de-4af9-bf0c-f4a2225d2bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:06.144009 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c79641-49de-4af9-bf0c-f4a2225d2bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:06.144287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c79641-49de-4af9-bf0c-f4a2225d2bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:06.145027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c79641-49de-4af9-bf0c-f4a2225d2bd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:06.145336 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c79641-49de-4af9-bf0c-f4a2225d2bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:06.146692 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c79641-49de-4af9-bf0c-f4a2225d2bd3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:06.154562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 633/1264] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:04:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:06.160940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd229428-4cd3-4504-884a-dffe05f4453f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:06.228293 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:06.260458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd229428-4cd3-4504-884a-dffe05f4453f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:06.293152 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf5db1-9be8-47b8-aa6e-9e642e5aa0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:06.293944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf5db1-9be8-47b8-aa6e-9e642e5aa0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:06.294252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf5db1-9be8-47b8-aa6e-9e642e5aa0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:06.294528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf5db1-9be8-47b8-aa6e-9e642e5aa0b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:06.305282 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf5db1-9be8-47b8-aa6e-9e642e5aa0b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:06.306733 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf5db1-9be8-47b8-aa6e-9e642e5aa0b6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:06.324389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-fedf5db1-9be8-47b8-aa6e-9e642e5aa0b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:04:06.324389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:06.324389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:06.324389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:06.324389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:06.324389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:06.324389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:06.324389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:06.324389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:06.324389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:06.324389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:06.324389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:06.324389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:04:06.324389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:04:06.324389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:06.325498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:06.326491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:04:06.326491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:04:06.326491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:04:06.326491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:04:06.326491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 632/1265] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:06.331988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c79641-49de-4af9-bf0c-f4a2225d2bd3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:06.412438 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:06.430982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c79641-49de-4af9-bf0c-f4a2225d2bd3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:06.464687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266e9ca-ddc6-48de-bd7b-1c35300942a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:06.465344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266e9ca-ddc6-48de-bd7b-1c35300942a4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:06.465632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266e9ca-ddc6-48de-bd7b-1c35300942a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:06.465912 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266e9ca-ddc6-48de-bd7b-1c35300942a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:06.466630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266e9ca-ddc6-48de-bd7b-1c35300942a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:06.466996 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266e9ca-ddc6-48de-bd7b-1c35300942a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:06.469161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266e9ca-ddc6-48de-bd7b-1c35300942a4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:06.480258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 634/1266] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:06.486203 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf5db1-9be8-47b8-aa6e-9e642e5aa0b6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:06.587139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf5db1-9be8-47b8-aa6e-9e642e5aa0b6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:06.605925 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:06.619660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63463c65-a763-4bd2-b236-3147027d235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:06.620466 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63463c65-a763-4bd2-b236-3147027d235e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:06.620759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63463c65-a763-4bd2-b236-3147027d235e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:06.620759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63463c65-a763-4bd2-b236-3147027d235e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:06.633435 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63463c65-a763-4bd2-b236-3147027d235e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:06.635640 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63463c65-a763-4bd2-b236-3147027d235e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-63463c65-a763-4bd2-b236-3147027d235e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:04:06.651291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:06.652588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:06.653776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:06.653776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:06.653776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:04:06.653776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:04:06.653776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:04:06.653776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:04:06.653776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:04:06.653776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:04:06.653776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 15:04:06.653776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:04:06.653776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 633/1267] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:04:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:06.658387 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266e9ca-ddc6-48de-bd7b-1c35300942a4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:06.765073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266e9ca-ddc6-48de-bd7b-1c35300942a4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:06.799487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fc531-94d2-4cad-a9a0-e1ba7c3d4051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:06.800217 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fc531-94d2-4cad-a9a0-e1ba7c3d4051 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:06.800503 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fc531-94d2-4cad-a9a0-e1ba7c3d4051 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:06.800777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fc531-94d2-4cad-a9a0-e1ba7c3d4051 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:06.801606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fc531-94d2-4cad-a9a0-e1ba7c3d4051 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:06.801911 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fc531-94d2-4cad-a9a0-e1ba7c3d4051 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:06.803392 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fc531-94d2-4cad-a9a0-e1ba7c3d4051 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:06.813135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:06.820948 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 635/1268] 10.222.0.67 () {56 vars in 1143 bytes} [Fri Jun 24 15:04:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:06.826668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63463c65-a763-4bd2-b236-3147027d235e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:06.927489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63463c65-a763-4bd2-b236-3147027d235e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:06.960854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0eed39-b25f-4caf-a76b-1bf499add131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:06.961732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eed39-b25f-4caf-a76b-1bf499add131 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:06.962011 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eed39-b25f-4caf-a76b-1bf499add131 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:06.962284 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eed39-b25f-4caf-a76b-1bf499add131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:06.972637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0eed39-b25f-4caf-a76b-1bf499add131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:06.974066 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0eed39-b25f-4caf-a76b-1bf499add131 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:06.991245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-7d0eed39-b25f-4caf-a76b-1bf499add131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:04:06.991245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:06.991245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:06.991245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:06.991245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:06.991245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:06.991245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:06.991245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:06.991245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:06.991245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:06.991245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:06.991245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:06.991245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:04:06.991245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:04:06.991245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:06.992417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:06.993605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:04:06.993605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:04:06.993605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:04:06.993605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:04:06.993605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 634/1269] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:06.998145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fc531-94d2-4cad-a9a0-e1ba7c3d4051 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:07.097341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fc531-94d2-4cad-a9a0-e1ba7c3d4051 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:07.132331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495c51d-7fa1-40b8-8aeb-045296057cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:07.133000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495c51d-7fa1-40b8-8aeb-045296057cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:07.133287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495c51d-7fa1-40b8-8aeb-045296057cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:07.133568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495c51d-7fa1-40b8-8aeb-045296057cf0 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:07.134296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495c51d-7fa1-40b8-8aeb-045296057cf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:07.134606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495c51d-7fa1-40b8-8aeb-045296057cf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:07.136050 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495c51d-7fa1-40b8-8aeb-045296057cf0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:07.147708 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 636/1270] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:07.153433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0eed39-b25f-4caf-a76b-1bf499add131 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:07.185951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:07.254286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0eed39-b25f-4caf-a76b-1bf499add131 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:07.288094 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50088384-b924-4066-87fb-53bcf1731b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:07.288896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50088384-b924-4066-87fb-53bcf1731b57 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:07.289183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50088384-b924-4066-87fb-53bcf1731b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:07.289464 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50088384-b924-4066-87fb-53bcf1731b57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:07.299998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50088384-b924-4066-87fb-53bcf1731b57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:07.301504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50088384-b924-4066-87fb-53bcf1731b57 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:07.315439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-50088384-b924-4066-87fb-53bcf1731b57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:04:07.315439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:07.315439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:07.315439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:07.315439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:07.315439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:07.315439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:07.315439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:07.315439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:07.315439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:07.315439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:07.315439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:07.315439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:04:07.315439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:04:07.315439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:07.316855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:07.317878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:04:07.317878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:04:07.317878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:04:07.317878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:04:07.317878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:04:07.317878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:04:07.317878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:04:07.317878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:04:07.317878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 635/1271] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:04:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:04:07.322564 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495c51d-7fa1-40b8-8aeb-045296057cf0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:07.379653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:07.421220 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495c51d-7fa1-40b8-8aeb-045296057cf0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:07.455572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783b76c-bae9-46ac-94c7-2cf794557751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:07.456205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783b76c-bae9-46ac-94c7-2cf794557751 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:07.456492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783b76c-bae9-46ac-94c7-2cf794557751 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:07.456806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783b76c-bae9-46ac-94c7-2cf794557751 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:07.457588 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783b76c-bae9-46ac-94c7-2cf794557751 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:07.457899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783b76c-bae9-46ac-94c7-2cf794557751 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:07.459340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783b76c-bae9-46ac-94c7-2cf794557751 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:07.471891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 637/1272] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:04:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:07.478083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50088384-b924-4066-87fb-53bcf1731b57 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:07.568235 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:07.576366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50088384-b924-4066-87fb-53bcf1731b57 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:07.610458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af0982-64d8-43aa-ad1f-9129f3692108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:07.611376 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af0982-64d8-43aa-ad1f-9129f3692108 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:07.611655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af0982-64d8-43aa-ad1f-9129f3692108 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:07.611923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af0982-64d8-43aa-ad1f-9129f3692108 None admin] PATH_INFO: `/v3/projects/06da054e8b7545dfa56780ab56585be8/users/f612d6ff06fc458e8c1178cbf8321f79/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:07.613829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af0982-64d8-43aa-ad1f-9129f3692108 None admin] RBAC: Authorizing `identity:create_grant(project_id=06da054e8b7545dfa56780ab56585be8, user_id=f612d6ff06fc458e8c1178cbf8321f79, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:07.615271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af0982-64d8-43aa-ad1f-9129f3692108 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:07.784854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 636/1273] 10.222.0.67 () {58 vars in 1439 bytes} [Fri Jun 24 15:04:07 2022] PUT /identity/v3/projects/06da054e8b7545dfa56780ab56585be8/users/f612d6ff06fc458e8c1178cbf8321f79/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:04:09.861502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf0b16-4197-401a-833a-8903bec3fa2c None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:09.862082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf0b16-4197-401a-833a-8903bec3fa2c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:09.862082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf0b16-4197-401a-833a-8903bec3fa2c None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:09.863477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 638/1274] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:04:09.868516 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf243e5-2219-4c93-8728-f9b566c96ca7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:09.868899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf243e5-2219-4c93-8728-f9b566c96ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:09.869091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf243e5-2219-4c93-8728-f9b566c96ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:09.886304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-abf243e5-2219-4c93-8728-f9b566c96ca7 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:04:09.887525 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-abf243e5-2219-4c93-8728-f9b566c96ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:04:09.953490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 637/1275] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:04:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:04:09.976993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf0b16-4197-401a-833a-8903bec3fa2c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:10.080241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf0b16-4197-401a-833a-8903bec3fa2c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:10.097926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:10.117209 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4de851-6c37-4b02-a9bd-0f671d65252e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:10.117209 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4de851-6c37-4b02-a9bd-0f671d65252e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:10.117209 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4de851-6c37-4b02-a9bd-0f671d65252e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:10.117209 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4de851-6c37-4b02-a9bd-0f671d65252e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:10.126539 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4de851-6c37-4b02-a9bd-0f671d65252e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-6e4de851-6c37-4b02-a9bd-0f671d65252e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 15:04:10.134163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:10.135358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:10.136462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:10.136462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:10.136462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:10.136462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:10.136462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 15:04:10.136462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 15:04:10.136462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 15:04:10.136462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 15:04:10.136462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 15:04:10.136462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 15:04:10.136462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 15:04:10.136462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:04:10.136462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 639/1276] 10.222.0.67 () {56 vars in 1096 bytes} [Fri Jun 24 15:04:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:10.141402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf243e5-2219-4c93-8728-f9b566c96ca7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:10.220183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:10.237934 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf243e5-2219-4c93-8728-f9b566c96ca7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:10.270548 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578469f-a766-4f34-8527-6abc52b3ed76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:10.271238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578469f-a766-4f34-8527-6abc52b3ed76 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:10.271526 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578469f-a766-4f34-8527-6abc52b3ed76 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:10.271804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578469f-a766-4f34-8527-6abc52b3ed76 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:10.272539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578469f-a766-4f34-8527-6abc52b3ed76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:10.272852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578469f-a766-4f34-8527-6abc52b3ed76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:10.274197 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578469f-a766-4f34-8527-6abc52b3ed76 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:10.282517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 638/1277] 10.222.0.67 () {56 vars in 1099 bytes} [Fri Jun 24 15:04:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:10.288138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4de851-6c37-4b02-a9bd-0f671d65252e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:10.385788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4de851-6c37-4b02-a9bd-0f671d65252e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:10.419021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7663c8-64a8-4ef3-9f46-e0235d52558f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:10.419825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7663c8-64a8-4ef3-9f46-e0235d52558f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:10.420107 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7663c8-64a8-4ef3-9f46-e0235d52558f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:10.420379 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7663c8-64a8-4ef3-9f46-e0235d52558f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:10.431450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7663c8-64a8-4ef3-9f46-e0235d52558f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:10.432964 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7663c8-64a8-4ef3-9f46-e0235d52558f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:10.449427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8e7663c8-64a8-4ef3-9f46-e0235d52558f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:04:10.449427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:10.449427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:10.449427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:10.449427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:10.449427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:10.449427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:10.449427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:10.449427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:10.449427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:10.449427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:10.449427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:10.449427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:04:10.449427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:04:10.449427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:10.450546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:10.457682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:04:10.457682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:04:10.457682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:04:10.457682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:04:10.457682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 640/1278] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:04:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:10.458054 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578469f-a766-4f34-8527-6abc52b3ed76 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:10.556109 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578469f-a766-4f34-8527-6abc52b3ed76 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:10.588497 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b795e7a-ea0d-4d09-a67c-c35efc8cc206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:10.589166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b795e7a-ea0d-4d09-a67c-c35efc8cc206 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:10.589559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b795e7a-ea0d-4d09-a67c-c35efc8cc206 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:10.589844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b795e7a-ea0d-4d09-a67c-c35efc8cc206 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:10.590566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b795e7a-ea0d-4d09-a67c-c35efc8cc206 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:10.590875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b795e7a-ea0d-4d09-a67c-c35efc8cc206 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:10.592273 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b795e7a-ea0d-4d09-a67c-c35efc8cc206 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:10.604990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 639/1279] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:10.611448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7663c8-64a8-4ef3-9f46-e0235d52558f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:10.643365 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:10.711723 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7663c8-64a8-4ef3-9f46-e0235d52558f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:10.745084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236690a5-cdf6-4b25-82ac-801b586e6ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:10.745879 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236690a5-cdf6-4b25-82ac-801b586e6ffa None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:10.746159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236690a5-cdf6-4b25-82ac-801b586e6ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:10.746431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236690a5-cdf6-4b25-82ac-801b586e6ffa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:10.753442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236690a5-cdf6-4b25-82ac-801b586e6ffa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:10.754867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236690a5-cdf6-4b25-82ac-801b586e6ffa None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-236690a5-cdf6-4b25-82ac-801b586e6ffa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 15:04:10.769522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:10.770667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:10.771884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:10.771884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:10.771884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 15:04:10.771884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 15:04:10.771884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 15:04:10.771884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 15:04:10.771884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 15:04:10.771884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 15:04:10.771884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 15:04:10.771884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:04:10.771884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 641/1280] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:04:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:10.776614 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b795e7a-ea0d-4d09-a67c-c35efc8cc206 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:10.780412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:10.875327 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b795e7a-ea0d-4d09-a67c-c35efc8cc206 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:10.909580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7cf43-62f1-4cf9-bb45-96288c058e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:10.910175 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7cf43-62f1-4cf9-bb45-96288c058e1a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:10.910453 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7cf43-62f1-4cf9-bb45-96288c058e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:10.910723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7cf43-62f1-4cf9-bb45-96288c058e1a None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:10.911571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7cf43-62f1-4cf9-bb45-96288c058e1a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:10.911875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7cf43-62f1-4cf9-bb45-96288c058e1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:10.913356 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7cf43-62f1-4cf9-bb45-96288c058e1a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:10.930120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 640/1281] 10.222.0.67 () {56 vars in 1143 bytes} [Fri Jun 24 15:04:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:10.935835 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236690a5-cdf6-4b25-82ac-801b586e6ffa None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:11.025323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:11.034914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236690a5-cdf6-4b25-82ac-801b586e6ffa None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:11.069204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4709f2-bc99-4f37-b2ef-de3c5c920335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:11.069854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4709f2-bc99-4f37-b2ef-de3c5c920335 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:11.070140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4709f2-bc99-4f37-b2ef-de3c5c920335 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:11.070418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4709f2-bc99-4f37-b2ef-de3c5c920335 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:11.139109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4709f2-bc99-4f37-b2ef-de3c5c920335 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:11.141185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4709f2-bc99-4f37-b2ef-de3c5c920335 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:11.227828 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:11.237812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 642/1282] 10.222.0.67 () {58 vars in 1282 bytes} [Fri Jun 24 15:04:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 15:04:11.243781 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7cf43-62f1-4cf9-bb45-96288c058e1a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:11.303622 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:11.341366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7cf43-62f1-4cf9-bb45-96288c058e1a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:11.373765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a164d7-5167-4bf2-aef2-3580e2571864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:11.375340 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a164d7-5167-4bf2-aef2-3580e2571864 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:11.375714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a164d7-5167-4bf2-aef2-3580e2571864 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:11.375993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a164d7-5167-4bf2-aef2-3580e2571864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:11.385816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a164d7-5167-4bf2-aef2-3580e2571864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:11.387275 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a164d7-5167-4bf2-aef2-3580e2571864 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:11.407002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-94a164d7-5167-4bf2-aef2-3580e2571864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:04:11.407002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:11.407002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:11.407002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:11.407002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:11.407002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:11.407002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:11.407002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:11.407002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:11.407002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:11.407002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:11.407002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:11.407002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 15:04:11.407002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 15:04:11.407002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:11.408122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:11.409060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 15:04:11.409060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 15:04:11.409060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 15:04:11.409060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:04:11.409291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 641/1283] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:04:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:11.415122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4709f2-bc99-4f37-b2ef-de3c5c920335 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:11.513280 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4709f2-bc99-4f37-b2ef-de3c5c920335 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:11.546916 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e963b-0b42-4636-b310-afe3611e4cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:11.547938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e963b-0b42-4636-b310-afe3611e4cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:11.548206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e963b-0b42-4636-b310-afe3611e4cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:11.548481 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e963b-0b42-4636-b310-afe3611e4cd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:11.549197 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e963b-0b42-4636-b310-afe3611e4cd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:11.549503 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e963b-0b42-4636-b310-afe3611e4cd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:11.550863 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e963b-0b42-4636-b310-afe3611e4cd3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:11.562453 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 643/1284] 10.222.0.67 () {56 vars in 1109 bytes} [Fri Jun 24 15:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:11.568111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a164d7-5167-4bf2-aef2-3580e2571864 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:11.665301 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a164d7-5167-4bf2-aef2-3580e2571864 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:11.679846 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:11.699461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66ca2d-c227-44a4-878b-cf1a5c27e1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:11.700219 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66ca2d-c227-44a4-878b-cf1a5c27e1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:11.700502 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66ca2d-c227-44a4-878b-cf1a5c27e1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:11.700777 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66ca2d-c227-44a4-878b-cf1a5c27e1a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:11.707334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66ca2d-c227-44a4-878b-cf1a5c27e1a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:11.708769 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66ca2d-c227-44a4-878b-cf1a5c27e1a1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:11.722518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-dd66ca2d-c227-44a4-878b-cf1a5c27e1a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:04:11.722518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:11.722518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:11.722518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:11.722518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:11.722518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:11.722518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:11.722518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:11.722518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:11.722518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:11.722518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:11.722518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:11.722518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 15:04:11.722518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 15:04:11.722518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:11.723655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:11.724650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 15:04:11.724650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 15:04:11.724650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 15:04:11.724650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 15:04:11.724650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 15:04:11.724650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 15:04:11.724650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 15:04:11.724650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jun 24 15:04:11.724650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 642/1285] 10.222.0.67 () {56 vars in 1111 bytes} [Fri Jun 24 15:04:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:11.729566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e963b-0b42-4636-b310-afe3611e4cd3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:11.778860 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:11.829205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e963b-0b42-4636-b310-afe3611e4cd3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:11.862943 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65716f56-bd9e-4765-9595-e259cff37247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:11.863631 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65716f56-bd9e-4765-9595-e259cff37247 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:11.863911 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65716f56-bd9e-4765-9595-e259cff37247 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:11.864179 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65716f56-bd9e-4765-9595-e259cff37247 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:11.864951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65716f56-bd9e-4765-9595-e259cff37247 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:11.865253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65716f56-bd9e-4765-9595-e259cff37247 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:11.866674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65716f56-bd9e-4765-9595-e259cff37247 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:11.878275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 644/1286] 10.222.0.67 () {56 vars in 1148 bytes} [Fri Jun 24 15:04:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:11.884317 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66ca2d-c227-44a4-878b-cf1a5c27e1a1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:11.985114 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66ca2d-c227-44a4-878b-cf1a5c27e1a1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:12.018149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba560a23-d52d-43a8-9fb8-8e187211aebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:12.018795 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba560a23-d52d-43a8-9fb8-8e187211aebd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:12.019080 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba560a23-d52d-43a8-9fb8-8e187211aebd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:12.019390 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba560a23-d52d-43a8-9fb8-8e187211aebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:12.020217 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba560a23-d52d-43a8-9fb8-8e187211aebd None admin] RBAC: Adding query filter params (role.id=6f8dcd4706ea44f1bcdcfd97ac90878d, scope.project.id=06da054e8b7545dfa56780ab56585be8, user.id=f612d6ff06fc458e8c1178cbf8321f79) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:12.020531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba560a23-d52d-43a8-9fb8-8e187211aebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:12.022001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba560a23-d52d-43a8-9fb8-8e187211aebd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:12.027715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 643/1287] 10.222.0.67 () {56 vars in 1374 bytes} [Fri Jun 24 15:04:11 2022] GET /identity/v3/role_assignments?user.id=f612d6ff06fc458e8c1178cbf8321f79&scope.project.id=06da054e8b7545dfa56780ab56585be8&role.id=6f8dcd4706ea44f1bcdcfd97ac90878d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:14.089041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6fef1-3b46-455f-8d80-72c1784b8d4f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:14.089565 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6fef1-3b46-455f-8d80-72c1784b8d4f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:14.089565 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6fef1-3b46-455f-8d80-72c1784b8d4f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:14.091131 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 645/1288] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:04:14.096256 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feeb83e-e9ab-40bd-8a0b-3ea0bc100439 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:14.096691 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feeb83e-e9ab-40bd-8a0b-3ea0bc100439 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:14.096806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feeb83e-e9ab-40bd-8a0b-3ea0bc100439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:14.113133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8feeb83e-e9ab-40bd-8a0b-3ea0bc100439 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:04:14.114296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8feeb83e-e9ab-40bd-8a0b-3ea0bc100439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:04:14.143607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:14.178836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 644/1289] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:04:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:04:14.202187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6fef1-3b46-455f-8d80-72c1784b8d4f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:14.301148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6fef1-3b46-455f-8d80-72c1784b8d4f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:14.311315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:14.334439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668cdd4-3a4a-4b83-a088-bec77f2f7691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:14.335249 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668cdd4-3a4a-4b83-a088-bec77f2f7691 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:14.335592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668cdd4-3a4a-4b83-a088-bec77f2f7691 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:14.335879 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668cdd4-3a4a-4b83-a088-bec77f2f7691 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:14.336520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668cdd4-3a4a-4b83-a088-bec77f2f7691 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:14.337881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668cdd4-3a4a-4b83-a088-bec77f2f7691 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-1668cdd4-3a4a-4b83-a088-bec77f2f7691 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:14.354056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:14.355142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:14.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:14.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:14.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:14.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:14.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:14.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:14.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:14.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:14.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:14.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:14.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:04:14.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:04:14.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 15:04:14.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:04:14.356182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 646/1290] 10.222.0.67 () {56 vars in 1117 bytes} [Fri Jun 24 15:04:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:14.357720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feeb83e-e9ab-40bd-8a0b-3ea0bc100439 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:14.457348 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feeb83e-e9ab-40bd-8a0b-3ea0bc100439 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:14.490845 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e328fea-f5a5-4061-85a0-e7a3b80ed45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:14.491540 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e328fea-f5a5-4061-85a0-e7a3b80ed45b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:14.491829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e328fea-f5a5-4061-85a0-e7a3b80ed45b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:14.492111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e328fea-f5a5-4061-85a0-e7a3b80ed45b None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:14.492813 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e328fea-f5a5-4061-85a0-e7a3b80ed45b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:14.493121 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e328fea-f5a5-4061-85a0-e7a3b80ed45b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:14.494493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e328fea-f5a5-4061-85a0-e7a3b80ed45b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:14.500013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 645/1291] 10.222.0.67 () {56 vars in 1116 bytes} [Fri Jun 24 15:04:14 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:04:14.505820 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668cdd4-3a4a-4b83-a088-bec77f2f7691 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:14.604495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668cdd4-3a4a-4b83-a088-bec77f2f7691 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:14.638094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdaedd-77e2-4ede-8269-84349b822659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:14.638746 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdaedd-77e2-4ede-8269-84349b822659 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:14.639139 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdaedd-77e2-4ede-8269-84349b822659 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:14.639379 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdaedd-77e2-4ede-8269-84349b822659 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:14.640086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdaedd-77e2-4ede-8269-84349b822659 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:14.640397 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdaedd-77e2-4ede-8269-84349b822659 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:14.641746 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdaedd-77e2-4ede-8269-84349b822659 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:14.646238 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 647/1292] 10.222.0.67 () {56 vars in 1116 bytes} [Fri Jun 24 15:04:14 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:16.645705 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7302f-9590-483e-a3e9-cf83d8b84570 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:16.646244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7302f-9590-483e-a3e9-cf83d8b84570 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:16.646244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7302f-9590-483e-a3e9-cf83d8b84570 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:16.647637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 646/1293] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:04:16.652718 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50abc4e-b833-499d-acee-5ac3e651cefe None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:16.653105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50abc4e-b833-499d-acee-5ac3e651cefe None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:16.653268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50abc4e-b833-499d-acee-5ac3e651cefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:16.670375 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c50abc4e-b833-499d-acee-5ac3e651cefe None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:04:16.671691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c50abc4e-b833-499d-acee-5ac3e651cefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:04:16.700137 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:16.737032 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 648/1294] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:04:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:04:16.759632 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7302f-9590-483e-a3e9-cf83d8b84570 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:16.790693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:16.858346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7302f-9590-483e-a3e9-cf83d8b84570 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:16.890623 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82202baf-1228-4c29-ad4d-0e2829ba43cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:16.891292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82202baf-1228-4c29-ad4d-0e2829ba43cb None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:16.891576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82202baf-1228-4c29-ad4d-0e2829ba43cb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:16.891852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82202baf-1228-4c29-ad4d-0e2829ba43cb None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:16.892554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82202baf-1228-4c29-ad4d-0e2829ba43cb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:16.893902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82202baf-1228-4c29-ad4d-0e2829ba43cb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:16.976973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 647/1295] 10.222.0.67 () {60 vars in 1141 bytes} [Fri Jun 24 15:04:16 2022] POST /identity/v3/services => generated 246 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:04:19.004247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567cad2-351e-408b-8ee1-b61736e81e06 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:19.004787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567cad2-351e-408b-8ee1-b61736e81e06 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:19.004917 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567cad2-351e-408b-8ee1-b61736e81e06 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:19.006219 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 649/1296] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:04:19.011630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961e03d-1fc9-45cf-b92a-177b26456083 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:19.012026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961e03d-1fc9-45cf-b92a-177b26456083 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:19.012199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961e03d-1fc9-45cf-b92a-177b26456083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:19.028605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f961e03d-1fc9-45cf-b92a-177b26456083 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:04:19.029790 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f961e03d-1fc9-45cf-b92a-177b26456083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:04:19.101932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 648/1297] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:04:19 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:04:19.126574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567cad2-351e-408b-8ee1-b61736e81e06 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:19.225647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567cad2-351e-408b-8ee1-b61736e81e06 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:19.259376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545626d5-9af1-462d-9982-64cf9095872b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:19.260106 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545626d5-9af1-462d-9982-64cf9095872b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:19.260388 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545626d5-9af1-462d-9982-64cf9095872b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:19.260662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545626d5-9af1-462d-9982-64cf9095872b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:19.261366 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545626d5-9af1-462d-9982-64cf9095872b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:19.262807 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545626d5-9af1-462d-9982-64cf9095872b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-545626d5-9af1-462d-9982-64cf9095872b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:19.275305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:19.276502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:19.277382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:19.277382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:19.277382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:19.277382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:19.277382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:19.277382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:19.277382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:19.277382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:19.277382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:19.277382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:19.277382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:04:19.277382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:04:19.277382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 15:04:19.277382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:04:19.277382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 650/1298] 10.222.0.67 () {56 vars in 1117 bytes} [Fri Jun 24 15:04:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 15:04:19.283409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961e03d-1fc9-45cf-b92a-177b26456083 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:19.302196 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:19.379515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961e03d-1fc9-45cf-b92a-177b26456083 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:19.411675 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c2214-d8bf-4343-9f51-0c79d746d795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:19.412333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c2214-d8bf-4343-9f51-0c79d746d795 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:19.412612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c2214-d8bf-4343-9f51-0c79d746d795 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:19.412887 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c2214-d8bf-4343-9f51-0c79d746d795 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:19.413649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c2214-d8bf-4343-9f51-0c79d746d795 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:19.413966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c2214-d8bf-4343-9f51-0c79d746d795 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:19.415340 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c2214-d8bf-4343-9f51-0c79d746d795 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:19.420267 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 649/1299] 10.222.0.67 () {56 vars in 1116 bytes} [Fri Jun 24 15:04:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:19.427314 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545626d5-9af1-462d-9982-64cf9095872b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:19.458718 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:19.524774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545626d5-9af1-462d-9982-64cf9095872b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:19.559135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b9507-7522-41c2-92a4-01ed47629142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:19.559818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b9507-7522-41c2-92a4-01ed47629142 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:19.560105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b9507-7522-41c2-92a4-01ed47629142 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:19.560385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b9507-7522-41c2-92a4-01ed47629142 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:19.561169 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b9507-7522-41c2-92a4-01ed47629142 None admin] RBAC: Adding query filter params (interface=public, service_id=bf33e28e6cf34a12be81977d8afe6bc7, region_id=RegionOne) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:19.561802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b9507-7522-41c2-92a4-01ed47629142 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:19.563329 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b9507-7522-41c2-92a4-01ed47629142 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:19.568333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 651/1300] 10.222.0.67 () {56 vars in 1251 bytes} [Fri Jun 24 15:04:19 2022] GET /identity/v3/endpoints?service_id=bf33e28e6cf34a12be81977d8afe6bc7&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:19.574084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c2214-d8bf-4343-9f51-0c79d746d795 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:19.656652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:19.673921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c2214-d8bf-4343-9f51-0c79d746d795 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:19.706975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae98866-8ce4-41d0-ad14-777d105758f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:19.707661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae98866-8ce4-41d0-ad14-777d105758f8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:19.707945 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae98866-8ce4-41d0-ad14-777d105758f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:19.708220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae98866-8ce4-41d0-ad14-777d105758f8 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:19.708887 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae98866-8ce4-41d0-ad14-777d105758f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:19.710246 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae98866-8ce4-41d0-ad14-777d105758f8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:19.716962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 650/1301] 10.222.0.67 () {56 vars in 1087 bytes} [Fri Jun 24 15:04:19 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:04:21.736988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5c2f8-db92-403b-8720-edfe0200f3fc None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:21.737587 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5c2f8-db92-403b-8720-edfe0200f3fc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:21.737587 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5c2f8-db92-403b-8720-edfe0200f3fc None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:21.738982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 652/1302] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:04:21.744062 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ee0c8-926e-4369-9bb6-5b168ba1d494 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:21.744460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ee0c8-926e-4369-9bb6-5b168ba1d494 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:21.744697 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ee0c8-926e-4369-9bb6-5b168ba1d494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:21.761130 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7f6ee0c8-926e-4369-9bb6-5b168ba1d494 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:04:21.762355 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7f6ee0c8-926e-4369-9bb6-5b168ba1d494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:04:21.828991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 651/1303] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:04:21 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 15:04:21.852573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5c2f8-db92-403b-8720-edfe0200f3fc None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:21.948105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:21.955254 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5c2f8-db92-403b-8720-edfe0200f3fc None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:21.988798 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7adc2-45cf-48fa-b9d3-36ee22090cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:21.989528 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7adc2-45cf-48fa-b9d3-36ee22090cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:21.989809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7adc2-45cf-48fa-b9d3-36ee22090cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:21.990081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7adc2-45cf-48fa-b9d3-36ee22090cc4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:21.990724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7adc2-45cf-48fa-b9d3-36ee22090cc4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:21.992293 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7adc2-45cf-48fa-b9d3-36ee22090cc4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e2c7adc2-45cf-48fa-b9d3-36ee22090cc4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 15:04:22.003968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 15:04:22.005191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 15:04:22.006086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 15:04:22.006086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jun 24 15:04:22.006086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 15:04:22.006086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jun 24 15:04:22.006086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 15:04:22.006086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 15:04:22.006086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 15:04:22.006086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jun 24 15:04:22.006086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 15:04:22.006086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jun 24 15:04:22.006086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 15:04:22.006086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 15:04:22.006086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 15:04:22.006086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jun 24 15:04:22.006086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 653/1304] 10.222.0.67 () {56 vars in 1117 bytes} [Fri Jun 24 15:04:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 15:04:22.011119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ee0c8-926e-4369-9bb6-5b168ba1d494 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:22.086348 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:22.109083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ee0c8-926e-4369-9bb6-5b168ba1d494 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:22.142520 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018ce1b-38d1-4cfd-be61-feff1c4a60df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:22.143204 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018ce1b-38d1-4cfd-be61-feff1c4a60df None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:22.143513 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018ce1b-38d1-4cfd-be61-feff1c4a60df None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:22.143795 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018ce1b-38d1-4cfd-be61-feff1c4a60df None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:22.144502 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018ce1b-38d1-4cfd-be61-feff1c4a60df None admin] RBAC: Adding query filter params (name=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:04:22.144814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018ce1b-38d1-4cfd-be61-feff1c4a60df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:22.146176 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018ce1b-38d1-4cfd-be61-feff1c4a60df None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:22.151280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 652/1305] 10.222.0.67 () {56 vars in 1116 bytes} [Fri Jun 24 15:04:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:04:22.157615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7adc2-45cf-48fa-b9d3-36ee22090cc4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:22.255255 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7adc2-45cf-48fa-b9d3-36ee22090cc4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:22.287826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9428ff3-98ff-451c-a520-4bd7363fbe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:22.288471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9428ff3-98ff-451c-a520-4bd7363fbe03 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:22.288752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9428ff3-98ff-451c-a520-4bd7363fbe03 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:22.289025 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9428ff3-98ff-451c-a520-4bd7363fbe03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:22.289845 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9428ff3-98ff-451c-a520-4bd7363fbe03 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:22.291250 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9428ff3-98ff-451c-a520-4bd7363fbe03 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:22.354023 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 654/1306] 10.222.0.67 () {60 vars in 1144 bytes} [Fri Jun 24 15:04:22 2022] POST /identity/v3/endpoints => generated 331 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:04:32.148281 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=30 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:32.349516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:24.173382 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a6a6d-e438-4c74-b065-5125c31f93e4 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:24.173968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a6a6d-e438-4c74-b065-5125c31f93e4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:24.173968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a6a6d-e438-4c74-b065-5125c31f93e4 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:24.175711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 653/1307] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:10:24.185896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7db819-5fe7-4189-b890-d75bc3b6e462 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:24.186446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7db819-5fe7-4189-b890-d75bc3b6e462 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:24.186722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7db819-5fe7-4189-b890-d75bc3b6e462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:24.310702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8f7db819-5fe7-4189-b890-d75bc3b6e462 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:10:24.312279 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8f7db819-5fe7-4189-b890-d75bc3b6e462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:24.528834 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 655/1308] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:10:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:10:24.590339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49430d37-bc19-485a-9f84-784c62fb384f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:24.590983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49430d37-bc19-485a-9f84-784c62fb384f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:24.591130 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49430d37-bc19-485a-9f84-784c62fb384f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:24.592606 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 654/1309] 10.222.0.67 () {54 vars in 964 bytes} [Fri Jun 24 15:10:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:10:24.609867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ac65a-7c14-470a-b42e-cbceb236f208 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:24.610315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ac65a-7c14-470a-b42e-cbceb236f208 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:24.610743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ac65a-7c14-470a-b42e-cbceb236f208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:24.767639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-cd4ac65a-7c14-470a-b42e-cbceb236f208 None None] MFA Rules not processed for user `95e7fb8f48dd4348b54e3694a8410bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:10:24.776229 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cd4ac65a-7c14-470a-b42e-cbceb236f208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:24.821521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:24.905889 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 656/1310] 10.222.0.67 () {58 vars in 1061 bytes} [Fri Jun 24 15:10:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:10:24.915513 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49430d37-bc19-485a-9f84-784c62fb384f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:10:24.920661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-49430d37-bc19-485a-9f84-784c62fb384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:25.055124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49430d37-bc19-485a-9f84-784c62fb384f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:10:25.141593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b16bc-1f39-4f48-8ac1-2c252fee1609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:10:25.142637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b16bc-1f39-4f48-8ac1-2c252fee1609 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:25.143073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b16bc-1f39-4f48-8ac1-2c252fee1609 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:25.143540 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b16bc-1f39-4f48-8ac1-2c252fee1609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:25.145370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1c1b16bc-1f39-4f48-8ac1-2c252fee1609 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:25.208928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:25.277477 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b16bc-1f39-4f48-8ac1-2c252fee1609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:10:25.279630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b16bc-1f39-4f48-8ac1-2c252fee1609 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:10:25.310698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 655/1311] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:10:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:10:27.717658 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05028236-83e8-4248-8b36-d41f89b813cd None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:27.718195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05028236-83e8-4248-8b36-d41f89b813cd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:27.718318 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05028236-83e8-4248-8b36-d41f89b813cd None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:27.719757 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 657/1312] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:10:27.725079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c26968-c2e4-457a-80cb-63ec95b8991f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:27.725490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c26968-c2e4-457a-80cb-63ec95b8991f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:27.725648 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c26968-c2e4-457a-80cb-63ec95b8991f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:27.810792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-38c26968-c2e4-457a-80cb-63ec95b8991f None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:10:27.812063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-38c26968-c2e4-457a-80cb-63ec95b8991f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:27.913766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 656/1313] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:10:27.953692 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05028236-83e8-4248-8b36-d41f89b813cd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:10:27.955248 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-05028236-83e8-4248-8b36-d41f89b813cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:28.058141 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05028236-83e8-4248-8b36-d41f89b813cd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:10:28.132806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:28.135374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586a24a-0264-46ba-805f-e67796cc39c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:10:28.136023 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586a24a-0264-46ba-805f-e67796cc39c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:28.136305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586a24a-0264-46ba-805f-e67796cc39c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:28.136783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586a24a-0264-46ba-805f-e67796cc39c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:28.138427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e586a24a-0264-46ba-805f-e67796cc39c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:28.248105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586a24a-0264-46ba-805f-e67796cc39c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:10:28.249555 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586a24a-0264-46ba-805f-e67796cc39c8 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:10:28.268591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 658/1314] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:10:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:10:30.928684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc107d3e-72ec-4445-b261-47416e520af2 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:30.929227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc107d3e-72ec-4445-b261-47416e520af2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:30.929227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc107d3e-72ec-4445-b261-47416e520af2 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:30.930596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 657/1315] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:10:30.936247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc6ab9-81db-4d00-be64-c3b85834204d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:30.936642 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc6ab9-81db-4d00-be64-c3b85834204d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:30.936801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc6ab9-81db-4d00-be64-c3b85834204d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:30.999491 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-17cc6ab9-81db-4d00-be64-c3b85834204d None None] MFA Rules not processed for user `2bf0569100d34ba7939616d4bcef590a`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:10:31.000693 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-17cc6ab9-81db-4d00-be64-c3b85834204d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:31.051414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:31.095726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 659/1316] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:10:31.135704 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc107d3e-72ec-4445-b261-47416e520af2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:10:31.137807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dc107d3e-72ec-4445-b261-47416e520af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:31.147452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:31.239775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc107d3e-72ec-4445-b261-47416e520af2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:10:31.312499 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b8b93-b40c-49a1-850b-879cffe64efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:10:31.313146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b8b93-b40c-49a1-850b-879cffe64efa service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:31.313432 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b8b93-b40c-49a1-850b-879cffe64efa service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:31.313709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b8b93-b40c-49a1-850b-879cffe64efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:31.315380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9e7b8b93-b40c-49a1-850b-879cffe64efa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:31.377715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:31.423447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b8b93-b40c-49a1-850b-879cffe64efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:10:31.424887 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b8b93-b40c-49a1-850b-879cffe64efa service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:10:31.443474 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 658/1317] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:10:31 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:10:34.211284 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d84e00-c33e-41db-96a2-49a33714a627 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:34.211722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d84e00-c33e-41db-96a2-49a33714a627 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:34.211824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d84e00-c33e-41db-96a2-49a33714a627 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:34.213205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 660/1318] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:10:34.218879 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fcd89-53c9-4ea7-a186-e08a094df148 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:34.219307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fcd89-53c9-4ea7-a186-e08a094df148 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:34.219472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fcd89-53c9-4ea7-a186-e08a094df148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:34.282858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-011fcd89-53c9-4ea7-a186-e08a094df148 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:10:34.284084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-011fcd89-53c9-4ea7-a186-e08a094df148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:34.378164 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 659/1319] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:10:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:10:34.421414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d84e00-c33e-41db-96a2-49a33714a627 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:10:34.422959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d3d84e00-c33e-41db-96a2-49a33714a627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:34.523946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d84e00-c33e-41db-96a2-49a33714a627 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:10:34.596071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6596a2e-5504-4e15-b449-4ba3d6f7f00e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:10:34.596713 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596a2e-5504-4e15-b449-4ba3d6f7f00e service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:34.596995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596a2e-5504-4e15-b449-4ba3d6f7f00e service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:34.597267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596a2e-5504-4e15-b449-4ba3d6f7f00e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:34.598900 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c6596a2e-5504-4e15-b449-4ba3d6f7f00e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:34.606147 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:34.705489 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6596a2e-5504-4e15-b449-4ba3d6f7f00e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:10:34.706925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6596a2e-5504-4e15-b449-4ba3d6f7f00e service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:10:34.725200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 661/1320] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:10:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:10:37.751363 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36fc28-d45a-46a0-8ba9-d9e4bba563a9 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:37.751955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36fc28-d45a-46a0-8ba9-d9e4bba563a9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:37.751955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36fc28-d45a-46a0-8ba9-d9e4bba563a9 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:37.753276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 660/1321] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:10:37.758418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d162775-5fb5-41ad-bc1a-4574eb3361a9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:37.758879 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d162775-5fb5-41ad-bc1a-4574eb3361a9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:37.758986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d162775-5fb5-41ad-bc1a-4574eb3361a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:37.823418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4d162775-5fb5-41ad-bc1a-4574eb3361a9 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:10:37.825152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4d162775-5fb5-41ad-bc1a-4574eb3361a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:37.882259 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:37.922110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 662/1322] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:10:37.958536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36fc28-d45a-46a0-8ba9-d9e4bba563a9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:10:37.960073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dc36fc28-d45a-46a0-8ba9-d9e4bba563a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:37.978846 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:38.062401 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36fc28-d45a-46a0-8ba9-d9e4bba563a9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:10:38.137383 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23500f0-f651-48be-ad6d-c3dd4291f6a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:10:38.138033 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23500f0-f651-48be-ad6d-c3dd4291f6a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:38.138320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23500f0-f651-48be-ad6d-c3dd4291f6a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:38.138596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23500f0-f651-48be-ad6d-c3dd4291f6a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:38.140272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a23500f0-f651-48be-ad6d-c3dd4291f6a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:38.213709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:38.256774 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23500f0-f651-48be-ad6d-c3dd4291f6a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:10:38.258218 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23500f0-f651-48be-ad6d-c3dd4291f6a8 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:10:38.277344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 661/1323] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:10:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:10:40.358591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db2c6b-986c-40ed-91ef-efe1860bd2bf None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:40.359131 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db2c6b-986c-40ed-91ef-efe1860bd2bf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:40.359131 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db2c6b-986c-40ed-91ef-efe1860bd2bf None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:40.360561 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 663/1324] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:10:40.366331 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3713b27-6142-4dd2-abb1-df4cf57a0da0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:40.376531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3713b27-6142-4dd2-abb1-df4cf57a0da0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:40.376531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3713b27-6142-4dd2-abb1-df4cf57a0da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:40.438986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f3713b27-6142-4dd2-abb1-df4cf57a0da0 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:10:40.440303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f3713b27-6142-4dd2-abb1-df4cf57a0da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:40.539396 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 662/1325] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:10:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:10:40.591157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111c659-9c7a-453b-a52c-83ae3a80f7e3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:40.591463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111c659-9c7a-453b-a52c-83ae3a80f7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:40.591737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111c659-9c7a-453b-a52c-83ae3a80f7e3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:40.593107 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 664/1326] 10.222.0.67 () {54 vars in 964 bytes} [Fri Jun 24 15:10:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:10:40.599551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816ffaa-6b4a-484e-8b71-0ab631bf6d10 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:40.599784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816ffaa-6b4a-484e-8b71-0ab631bf6d10 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:40.600063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816ffaa-6b4a-484e-8b71-0ab631bf6d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:40.630650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:40.675930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3816ffaa-6b4a-484e-8b71-0ab631bf6d10 None None] MFA Rules not processed for user `95e7fb8f48dd4348b54e3694a8410bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:10:40.677149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3816ffaa-6b4a-484e-8b71-0ab631bf6d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:40.754595 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 663/1327] 10.222.0.67 () {58 vars in 1061 bytes} [Fri Jun 24 15:10:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:10:40.763418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111c659-9c7a-453b-a52c-83ae3a80f7e3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:10:40.764931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5111c659-9c7a-453b-a52c-83ae3a80f7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:40.869552 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111c659-9c7a-453b-a52c-83ae3a80f7e3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:10:40.942069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1761a1c-633c-4e51-a9e1-46bc03b6bc7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:10:40.942715 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1761a1c-633c-4e51-a9e1-46bc03b6bc7e service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:40.942997 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1761a1c-633c-4e51-a9e1-46bc03b6bc7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:40.943309 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1761a1c-633c-4e51-a9e1-46bc03b6bc7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:40.944930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a1761a1c-633c-4e51-a9e1-46bc03b6bc7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:40.961281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:41.051931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1761a1c-633c-4e51-a9e1-46bc03b6bc7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:10:41.053370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1761a1c-633c-4e51-a9e1-46bc03b6bc7e service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:10:41.071395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 665/1328] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:10:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:10:43.410698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaceaf6-91ea-47d6-a77e-84a0887935f6 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:43.411299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaceaf6-91ea-47d6-a77e-84a0887935f6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:43.411299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaceaf6-91ea-47d6-a77e-84a0887935f6 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:43.412857 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 664/1329] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:10:43.418572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b4308-04de-4dfb-a5d2-66044a3830bc None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:43.418986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b4308-04de-4dfb-a5d2-66044a3830bc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:43.419185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b4308-04de-4dfb-a5d2-66044a3830bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:43.487421 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f30b4308-04de-4dfb-a5d2-66044a3830bc None None] MFA Rules not processed for user `2bf0569100d34ba7939616d4bcef590a`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:10:43.488625 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f30b4308-04de-4dfb-a5d2-66044a3830bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:43.555943 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:43.588892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 666/1330] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:10:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:10:43.628006 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaceaf6-91ea-47d6-a77e-84a0887935f6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:10:43.629547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-caaceaf6-91ea-47d6-a77e-84a0887935f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:43.733596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaceaf6-91ea-47d6-a77e-84a0887935f6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:10:43.754859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:43.810326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25818172-b871-4c0f-a5f3-20b265f5a2e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:10:43.810980 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25818172-b871-4c0f-a5f3-20b265f5a2e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:43.811294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25818172-b871-4c0f-a5f3-20b265f5a2e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:43.811576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25818172-b871-4c0f-a5f3-20b265f5a2e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:43.813215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-25818172-b871-4c0f-a5f3-20b265f5a2e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:43.922792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25818172-b871-4c0f-a5f3-20b265f5a2e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:10:43.926572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25818172-b871-4c0f-a5f3-20b265f5a2e5 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:10:43.950517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 665/1331] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:10:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:10:46.647154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571d8cf-8bb7-42e6-8fcb-51fae11f55b2 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:46.647701 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571d8cf-8bb7-42e6-8fcb-51fae11f55b2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:46.647701 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571d8cf-8bb7-42e6-8fcb-51fae11f55b2 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:46.649053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 667/1332] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:10:46.654861 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cbacf-270b-4929-8f5e-ecdbc4d53c28 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:46.655310 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cbacf-270b-4929-8f5e-ecdbc4d53c28 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:46.655464 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cbacf-270b-4929-8f5e-ecdbc4d53c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:46.720645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:46.723239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2c8cbacf-270b-4929-8f5e-ecdbc4d53c28 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:10:46.724465 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2c8cbacf-270b-4929-8f5e-ecdbc4d53c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:46.819620 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 666/1333] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:10:46.856030 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571d8cf-8bb7-42e6-8fcb-51fae11f55b2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:10:46.857571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e571d8cf-8bb7-42e6-8fcb-51fae11f55b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:46.957533 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571d8cf-8bb7-42e6-8fcb-51fae11f55b2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:10:47.030719 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b696798-b20a-4ed8-928f-1334feb4f7f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:10:47.031383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b696798-b20a-4ed8-928f-1334feb4f7f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:47.031663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b696798-b20a-4ed8-928f-1334feb4f7f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:47.031934 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b696798-b20a-4ed8-928f-1334feb4f7f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:47.033540 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3b696798-b20a-4ed8-928f-1334feb4f7f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:47.059091 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:47.140172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b696798-b20a-4ed8-928f-1334feb4f7f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:10:47.141613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b696798-b20a-4ed8-928f-1334feb4f7f7 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:10:47.160338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 668/1334] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:10:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:10:49.291283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89845f98-68ab-46eb-80f1-0438af3729f6 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:49.292104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89845f98-68ab-46eb-80f1-0438af3729f6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:49.292213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89845f98-68ab-46eb-80f1-0438af3729f6 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:49.294353 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 667/1335] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:10:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:10:49.300005 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff9873-d647-42de-9065-a6811c9d9cf3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:49.300454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff9873-d647-42de-9065-a6811c9d9cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:49.300572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff9873-d647-42de-9065-a6811c9d9cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:49.377674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c3ff9873-d647-42de-9065-a6811c9d9cf3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:10:49.378798 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c3ff9873-d647-42de-9065-a6811c9d9cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:49.466423 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:49.503424 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 669/1336] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:10:49.529142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89845f98-68ab-46eb-80f1-0438af3729f6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:10:49.530693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-89845f98-68ab-46eb-80f1-0438af3729f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:49.637979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89845f98-68ab-46eb-80f1-0438af3729f6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:10:49.667477 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:49.712527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88167e7-3177-4e82-b986-66a2dc47e8e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:10:49.713181 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88167e7-3177-4e82-b986-66a2dc47e8e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:49.713462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88167e7-3177-4e82-b986-66a2dc47e8e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:49.713737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88167e7-3177-4e82-b986-66a2dc47e8e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:49.716829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f88167e7-3177-4e82-b986-66a2dc47e8e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:49.846676 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88167e7-3177-4e82-b986-66a2dc47e8e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:10:49.849476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88167e7-3177-4e82-b986-66a2dc47e8e2 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:10:49.873096 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 668/1337] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:10:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:10:52.629090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6110314-ceee-4d75-bf99-d1a2abe667b3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:52.629627 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6110314-ceee-4d75-bf99-d1a2abe667b3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:52.629727 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6110314-ceee-4d75-bf99-d1a2abe667b3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:52.631036 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 670/1338] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:10:52.636217 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0879099-dab9-4715-b55d-c5372e3dc32e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:52.637333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0879099-dab9-4715-b55d-c5372e3dc32e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:52.637618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0879099-dab9-4715-b55d-c5372e3dc32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:52.700478 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b0879099-dab9-4715-b55d-c5372e3dc32e None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:10:52.701686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b0879099-dab9-4715-b55d-c5372e3dc32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:52.709881 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:52.796112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 669/1339] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:10:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:10:52.832664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6110314-ceee-4d75-bf99-d1a2abe667b3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:10:52.834202 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d6110314-ceee-4d75-bf99-d1a2abe667b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:52.936245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6110314-ceee-4d75-bf99-d1a2abe667b3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:10:53.007342 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a7496-f092-486a-a9f4-b9a0cfbf4f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:10:53.007985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a7496-f092-486a-a9f4-b9a0cfbf4f65 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:53.008263 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a7496-f092-486a-a9f4-b9a0cfbf4f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:53.008536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a7496-f092-486a-a9f4-b9a0cfbf4f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:53.010156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4a5a7496-f092-486a-a9f4-b9a0cfbf4f65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:53.046411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:53.118480 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a7496-f092-486a-a9f4-b9a0cfbf4f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:10:53.119946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a7496-f092-486a-a9f4-b9a0cfbf4f65 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:10:53.138943 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 671/1340] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:10:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:10:57.377331 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d1c25-a066-428d-8e10-93a50197dfa3 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:57.377817 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d1c25-a066-428d-8e10-93a50197dfa3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:57.377937 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d1c25-a066-428d-8e10-93a50197dfa3 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:57.379342 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 670/1341] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:10:57.384902 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848816b-31f5-42a5-b7f1-dd498f8d0e50 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:57.385310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848816b-31f5-42a5-b7f1-dd498f8d0e50 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:57.385490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848816b-31f5-42a5-b7f1-dd498f8d0e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:57.471527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4848816b-31f5-42a5-b7f1-dd498f8d0e50 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:10:57.472909 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4848816b-31f5-42a5-b7f1-dd498f8d0e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:57.572786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:57.586150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 672/1342] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:10:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:10:57.625789 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d1c25-a066-428d-8e10-93a50197dfa3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:10:57.628185 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ca3d1c25-a066-428d-8e10-93a50197dfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:57.727254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d1c25-a066-428d-8e10-93a50197dfa3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:10:57.730782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053155f3-842b-449f-bfac-6a467c289553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:10:57.731462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053155f3-842b-449f-bfac-6a467c289553 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:57.731747 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053155f3-842b-449f-bfac-6a467c289553 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:57.732156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053155f3-842b-449f-bfac-6a467c289553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:57.741272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053155f3-842b-449f-bfac-6a467c289553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:10:57.742740 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053155f3-842b-449f-bfac-6a467c289553 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:10:57.752308 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 671/1343] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:10:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:00.707920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87d62e-6bd6-48d0-9c5f-0a97a0058394 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:00.708463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87d62e-6bd6-48d0-9c5f-0a97a0058394 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:00.708463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87d62e-6bd6-48d0-9c5f-0a97a0058394 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:00.709884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 673/1344] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:00.716675 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4468f6e-9549-4e53-9dad-fa096c112725 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:00.717240 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4468f6e-9549-4e53-9dad-fa096c112725 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:00.717240 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4468f6e-9549-4e53-9dad-fa096c112725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:00.748617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e4468f6e-9549-4e53-9dad-fa096c112725 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:00.749829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e4468f6e-9549-4e53-9dad-fa096c112725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:00.780607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:11:00.832830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 672/1345] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:11:00.872435 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87d62e-6bd6-48d0-9c5f-0a97a0058394 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:00.883323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87d62e-6bd6-48d0-9c5f-0a97a0058394 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:00.887066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9cbfc-cb72-4f31-a0a8-b2fc12d5e1b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:00.887776 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9cbfc-cb72-4f31-a0a8-b2fc12d5e1b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:00.888064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9cbfc-cb72-4f31-a0a8-b2fc12d5e1b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:00.888340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9cbfc-cb72-4f31-a0a8-b2fc12d5e1b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:00.897438 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9cbfc-cb72-4f31-a0a8-b2fc12d5e1b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:00.898875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9cbfc-cb72-4f31-a0a8-b2fc12d5e1b6 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:00.909100 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 674/1346] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:05.203074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b85ba-94c3-469e-9ee9-6614dcfa78ac None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:05.203642 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b85ba-94c3-469e-9ee9-6614dcfa78ac None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:05.203761 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b85ba-94c3-469e-9ee9-6614dcfa78ac None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:05.205087 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 673/1347] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:05.210342 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a57a1-8368-4bc0-80fb-db98b5991a63 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:05.210764 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a57a1-8368-4bc0-80fb-db98b5991a63 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:05.210944 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a57a1-8368-4bc0-80fb-db98b5991a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:05.237377 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e90a57a1-8368-4bc0-80fb-db98b5991a63 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:05.239513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e90a57a1-8368-4bc0-80fb-db98b5991a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:05.253887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 675/1348] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:05.291062 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b85ba-94c3-469e-9ee9-6614dcfa78ac None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:05.302692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b85ba-94c3-469e-9ee9-6614dcfa78ac None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:05.306739 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910bcee-e806-4e76-ace0-6395f66f48ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:05.307414 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910bcee-e806-4e76-ace0-6395f66f48ad service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:05.307704 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910bcee-e806-4e76-ace0-6395f66f48ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:05.308557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910bcee-e806-4e76-ace0-6395f66f48ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:05.317580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910bcee-e806-4e76-ace0-6395f66f48ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:05.319017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910bcee-e806-4e76-ace0-6395f66f48ad service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:05.328911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 674/1349] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:10.377838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cb05d-4d00-4e91-8803-5c0150f41275 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:10.377838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cb05d-4d00-4e91-8803-5c0150f41275 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:10.377838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cb05d-4d00-4e91-8803-5c0150f41275 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:10.378722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 676/1350] 10.222.0.67 () {54 vars in 921 bytes} [Fri Jun 24 15:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:10.382447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce279c-265f-42a5-9960-0860219d1037 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:10.382860 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce279c-265f-42a5-9960-0860219d1037 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:10.383036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce279c-265f-42a5-9960-0860219d1037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:10.458038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ddce279c-265f-42a5-9960-0860219d1037 None None] MFA Rules not processed for user `f612d6ff06fc458e8c1178cbf8321f79`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:10.459289 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ddce279c-265f-42a5-9960-0860219d1037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:10.546551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 675/1351] 10.222.0.67 () {58 vars in 1018 bytes} [Fri Jun 24 15:11:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:14.721034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea98e74-57d5-43db-8d94-4112f513ecf6 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:14.721572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea98e74-57d5-43db-8d94-4112f513ecf6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:14.721572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea98e74-57d5-43db-8d94-4112f513ecf6 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:14.723259 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 677/1352] 10.222.0.67 () {54 vars in 921 bytes} [Fri Jun 24 15:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:14.728988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c619a7-f5f9-4248-bb64-308e3017496e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:14.730146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c619a7-f5f9-4248-bb64-308e3017496e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:14.730560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c619a7-f5f9-4248-bb64-308e3017496e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:14.749174 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c5c619a7-f5f9-4248-bb64-308e3017496e None None] MFA Rules not processed for user `f612d6ff06fc458e8c1178cbf8321f79`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:14.750454 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c5c619a7-f5f9-4248-bb64-308e3017496e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:14.759238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 676/1353] 10.222.0.67 () {58 vars in 1018 bytes} [Fri Jun 24 15:11:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:15.233137 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=13 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:11:16.227117 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc367a6-cb61-4965-8018-da349ae0a352 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:16.227729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc367a6-cb61-4965-8018-da349ae0a352 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:16.228017 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc367a6-cb61-4965-8018-da349ae0a352 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:16.229301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 678/1354] 10.222.0.67 () {54 vars in 921 bytes} [Fri Jun 24 15:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:16.234412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98984f27-b8c7-4a43-ab84-4a9a25ec0d9c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:16.234859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98984f27-b8c7-4a43-ab84-4a9a25ec0d9c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:16.235101 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98984f27-b8c7-4a43-ab84-4a9a25ec0d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:16.253995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-98984f27-b8c7-4a43-ab84-4a9a25ec0d9c None None] MFA Rules not processed for user `f612d6ff06fc458e8c1178cbf8321f79`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:16.255210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-98984f27-b8c7-4a43-ab84-4a9a25ec0d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:16.264458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 677/1355] 10.222.0.67 () {58 vars in 1018 bytes} [Fri Jun 24 15:11:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:11:19.912000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8a944-249f-43ed-9dab-c6749844d534 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:19.912000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8a944-249f-43ed-9dab-c6749844d534 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:19.912609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8a944-249f-43ed-9dab-c6749844d534 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:19.914011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 679/1356] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:11:19.922561 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1328b-d158-483f-b254-4d759629fdfe None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:19.923397 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1328b-d158-483f-b254-4d759629fdfe None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:19.924592 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1328b-d158-483f-b254-4d759629fdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:19.945342 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-73b1328b-d158-483f-b254-4d759629fdfe None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:19.946918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-73b1328b-d158-483f-b254-4d759629fdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:19.956674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 678/1357] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:20.013947 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8353c-695a-48a1-8ccc-66dfd2bd22fc None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:20.014108 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8353c-695a-48a1-8ccc-66dfd2bd22fc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:20.014523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8353c-695a-48a1-8ccc-66dfd2bd22fc None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:20.016057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 680/1358] 10.222.0.67 () {54 vars in 961 bytes} [Fri Jun 24 15:11:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:20.023543 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13091da8-4202-4468-a4e9-39adb2fbd3bf None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:20.023835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13091da8-4202-4468-a4e9-39adb2fbd3bf None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:20.024114 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13091da8-4202-4468-a4e9-39adb2fbd3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:20.070594 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:11:20.079615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-13091da8-4202-4468-a4e9-39adb2fbd3bf None None] MFA Rules not processed for user `51de59caa3144a1cbf12b20b15e2134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:20.080899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-13091da8-4202-4468-a4e9-39adb2fbd3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:20.170461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 679/1359] 10.222.0.67 () {58 vars in 1058 bytes} [Fri Jun 24 15:11:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:20.179260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8353c-695a-48a1-8ccc-66dfd2bd22fc None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:20.194178 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8353c-695a-48a1-8ccc-66dfd2bd22fc None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:20.201778 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c87cd3-f7e6-4f1d-9465-e5bd9c2ff0e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:20.202511 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c87cd3-f7e6-4f1d-9465-e5bd9c2ff0e4 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:20.203022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c87cd3-f7e6-4f1d-9465-e5bd9c2ff0e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:20.203510 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c87cd3-f7e6-4f1d-9465-e5bd9c2ff0e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:20.212991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c87cd3-f7e6-4f1d-9465-e5bd9c2ff0e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:20.215496 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c87cd3-f7e6-4f1d-9465-e5bd9c2ff0e4 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:20.225753 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 681/1360] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:22.492563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf42980-60f6-491e-b9d6-bbda661cdef4 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:22.493068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf42980-60f6-491e-b9d6-bbda661cdef4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:22.493068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf42980-60f6-491e-b9d6-bbda661cdef4 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:22.494559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 680/1361] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:22.499971 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc5b63-99b2-4440-8d54-d99dcf6aa07d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:22.500375 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc5b63-99b2-4440-8d54-d99dcf6aa07d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:22.500522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc5b63-99b2-4440-8d54-d99dcf6aa07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:22.519419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f7fc5b63-99b2-4440-8d54-d99dcf6aa07d None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:22.520604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f7fc5b63-99b2-4440-8d54-d99dcf6aa07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:22.529223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 682/1362] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:22.574065 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf42980-60f6-491e-b9d6-bbda661cdef4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:22.595711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf42980-60f6-491e-b9d6-bbda661cdef4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:22.601102 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01372e1-bd17-4c92-b440-e7ddc7ae7d5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:22.601771 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01372e1-bd17-4c92-b440-e7ddc7ae7d5c service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:22.602058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01372e1-bd17-4c92-b440-e7ddc7ae7d5c service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:22.602336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01372e1-bd17-4c92-b440-e7ddc7ae7d5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:22.611567 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01372e1-bd17-4c92-b440-e7ddc7ae7d5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:22.613765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01372e1-bd17-4c92-b440-e7ddc7ae7d5c service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:22.624728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 681/1363] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:25.176553 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47cc2b-1b92-4d2b-a3a2-177d8a2877fe None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:25.177101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47cc2b-1b92-4d2b-a3a2-177d8a2877fe None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:25.177101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47cc2b-1b92-4d2b-a3a2-177d8a2877fe None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:25.178488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 683/1364] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:25.184065 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7a687-460e-4778-a374-7a219b70a4ca None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:25.184503 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7a687-460e-4778-a374-7a219b70a4ca None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:25.184719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7a687-460e-4778-a374-7a219b70a4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:25.202637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-63e7a687-460e-4778-a374-7a219b70a4ca None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:25.203954 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-63e7a687-460e-4778-a374-7a219b70a4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:25.211775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 682/1365] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:25.259821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276f453-7846-491d-adcb-a5c1719bb0e2 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:25.260686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276f453-7846-491d-adcb-a5c1719bb0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:25.260943 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276f453-7846-491d-adcb-a5c1719bb0e2 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:25.262500 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 684/1366] 10.222.0.67 () {54 vars in 961 bytes} [Fri Jun 24 15:11:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:25.267907 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5889c7-8508-48bd-aefd-421e4a3fd7ba None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:25.268196 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5889c7-8508-48bd-aefd-421e4a3fd7ba None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:25.268571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5889c7-8508-48bd-aefd-421e4a3fd7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:25.287147 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0d5889c7-8508-48bd-aefd-421e4a3fd7ba None None] MFA Rules not processed for user `51de59caa3144a1cbf12b20b15e2134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:25.288414 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0d5889c7-8508-48bd-aefd-421e4a3fd7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:25.298627 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 683/1367] 10.222.0.67 () {58 vars in 1058 bytes} [Fri Jun 24 15:11:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:25.306219 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276f453-7846-491d-adcb-a5c1719bb0e2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:25.320572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276f453-7846-491d-adcb-a5c1719bb0e2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:25.326531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056e2cd-a227-4d15-a0be-58dd8237f131 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:25.327167 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056e2cd-a227-4d15-a0be-58dd8237f131 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:25.327479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056e2cd-a227-4d15-a0be-58dd8237f131 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:25.327759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056e2cd-a227-4d15-a0be-58dd8237f131 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:25.335988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056e2cd-a227-4d15-a0be-58dd8237f131 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:25.337574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056e2cd-a227-4d15-a0be-58dd8237f131 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:25.348107 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 685/1368] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:27.581355 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad834c1-1b42-42cc-a1a4-14349f8c4eec None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:27.582239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad834c1-1b42-42cc-a1a4-14349f8c4eec None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:27.582520 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad834c1-1b42-42cc-a1a4-14349f8c4eec None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:27.584092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 684/1369] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:27.589329 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a11ad8-25b2-4405-934b-cf29e3afb7ee None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:27.589759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a11ad8-25b2-4405-934b-cf29e3afb7ee None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:27.590051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a11ad8-25b2-4405-934b-cf29e3afb7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:27.608599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b2a11ad8-25b2-4405-934b-cf29e3afb7ee None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:27.609777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b2a11ad8-25b2-4405-934b-cf29e3afb7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:27.618166 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 686/1370] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:27.665402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad834c1-1b42-42cc-a1a4-14349f8c4eec None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:27.681396 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad834c1-1b42-42cc-a1a4-14349f8c4eec None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:27.687986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934baed2-39be-4760-80ae-478fe016f099 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:27.688642 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934baed2-39be-4760-80ae-478fe016f099 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:27.688923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934baed2-39be-4760-80ae-478fe016f099 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:27.689200 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934baed2-39be-4760-80ae-478fe016f099 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:27.697720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934baed2-39be-4760-80ae-478fe016f099 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:27.699329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934baed2-39be-4760-80ae-478fe016f099 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:27.708803 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 685/1371] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:28.220715 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfd200-5872-428a-90b4-6c4533b69d73 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:28.221286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfd200-5872-428a-90b4-6c4533b69d73 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:28.221582 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfd200-5872-428a-90b4-6c4533b69d73 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:28.223121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 687/1372] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:28.228583 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecded80-d857-433a-a193-db0c7a9d322a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:28.229619 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecded80-d857-433a-a193-db0c7a9d322a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:28.230211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecded80-d857-433a-a193-db0c7a9d322a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:28.249383 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-eecded80-d857-433a-a193-db0c7a9d322a None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:28.250695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-eecded80-d857-433a-a193-db0c7a9d322a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:28.259228 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 686/1373] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:28.614274 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c2b45-1115-4e28-836b-9264100d9afe None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:28.614274 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c2b45-1115-4e28-836b-9264100d9afe None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:28.614274 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c2b45-1115-4e28-836b-9264100d9afe None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:28.616651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 688/1374] 10.222.0.67 () {54 vars in 963 bytes} [Fri Jun 24 15:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:28.620754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085be12-d9fd-455e-82d4-a82865c630d3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:28.621243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085be12-d9fd-455e-82d4-a82865c630d3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:28.621397 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085be12-d9fd-455e-82d4-a82865c630d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:28.674959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5085be12-d9fd-455e-82d4-a82865c630d3 None None] MFA Rules not processed for user `2efb120c084a48858d32f97843d76834`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:28.676206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5085be12-d9fd-455e-82d4-a82865c630d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:28.746018 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 687/1375] 10.222.0.67 () {58 vars in 1060 bytes} [Fri Jun 24 15:11:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:28.754251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c2b45-1115-4e28-836b-9264100d9afe None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:28.765944 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c2b45-1115-4e28-836b-9264100d9afe None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:28.770237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fd72f-f758-44ae-8e53-71d6e4215e06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:28.771152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fd72f-f758-44ae-8e53-71d6e4215e06 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:28.776475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fd72f-f758-44ae-8e53-71d6e4215e06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:28.776475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fd72f-f758-44ae-8e53-71d6e4215e06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:28.783361 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fd72f-f758-44ae-8e53-71d6e4215e06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:28.784815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fd72f-f758-44ae-8e53-71d6e4215e06 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:28.796237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 689/1376] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:30.038961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc7096-6e91-493f-8564-f598367461e9 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:30.039545 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc7096-6e91-493f-8564-f598367461e9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:30.039545 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc7096-6e91-493f-8564-f598367461e9 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:30.041207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 688/1377] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:30.047049 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f9fc2-1073-49d9-ade5-1cb79daf236e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:30.047472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f9fc2-1073-49d9-ade5-1cb79daf236e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:30.047665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f9fc2-1073-49d9-ade5-1cb79daf236e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:30.065388 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a34f9fc2-1073-49d9-ade5-1cb79daf236e None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:30.066580 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a34f9fc2-1073-49d9-ade5-1cb79daf236e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:30.074365 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 690/1378] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:30.119349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc7096-6e91-493f-8564-f598367461e9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:30.134389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc7096-6e91-493f-8564-f598367461e9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:30.140096 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79542c-5d2f-4c27-a29e-d4b01af1f5ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:30.140735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79542c-5d2f-4c27-a29e-d4b01af1f5ad service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:30.141025 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79542c-5d2f-4c27-a29e-d4b01af1f5ad service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:30.141304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79542c-5d2f-4c27-a29e-d4b01af1f5ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:30.149543 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b79542c-5d2f-4c27-a29e-d4b01af1f5ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:30.151937 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b79542c-5d2f-4c27-a29e-d4b01af1f5ad service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:30.155975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:11:30.162178 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 689/1379] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:32.375220 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c6471-bb44-4347-bf58-7225a4f5384d None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:32.376073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c6471-bb44-4347-bf58-7225a4f5384d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:32.376524 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c6471-bb44-4347-bf58-7225a4f5384d None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:32.382968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 691/1380] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:32.384533 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7f285-a039-4f46-9df9-e8813192d6a8 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:32.385006 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7f285-a039-4f46-9df9-e8813192d6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:32.385289 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7f285-a039-4f46-9df9-e8813192d6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:32.408440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-39b7f285-a039-4f46-9df9-e8813192d6a8 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:32.411215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-39b7f285-a039-4f46-9df9-e8813192d6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:32.421767 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 690/1381] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:32.468064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c6471-bb44-4347-bf58-7225a4f5384d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:32.484171 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c6471-bb44-4347-bf58-7225a4f5384d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:32.490024 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c79a35-0f8d-4647-982c-dea1fea539db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:32.490863 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c79a35-0f8d-4647-982c-dea1fea539db service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:32.491364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c79a35-0f8d-4647-982c-dea1fea539db service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:32.491796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c79a35-0f8d-4647-982c-dea1fea539db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:32.501199 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c79a35-0f8d-4647-982c-dea1fea539db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:32.503003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c79a35-0f8d-4647-982c-dea1fea539db service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:32.512829 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 692/1382] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:34.756002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22165bb2-91c4-4950-98b6-aa3cbb69b012 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:34.756533 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22165bb2-91c4-4950-98b6-aa3cbb69b012 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:34.756533 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22165bb2-91c4-4950-98b6-aa3cbb69b012 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:34.757932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 691/1383] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:34.763430 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12209191-a35f-4017-9df7-f07328f7c287 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:34.763885 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12209191-a35f-4017-9df7-f07328f7c287 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:34.763998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12209191-a35f-4017-9df7-f07328f7c287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:34.782667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-12209191-a35f-4017-9df7-f07328f7c287 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:34.783808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-12209191-a35f-4017-9df7-f07328f7c287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:34.804925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 693/1384] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:34.834933 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22165bb2-91c4-4950-98b6-aa3cbb69b012 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:34.850922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22165bb2-91c4-4950-98b6-aa3cbb69b012 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:34.858025 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cf92a-e57d-4879-b189-936724796f62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:34.858674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cf92a-e57d-4879-b189-936724796f62 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:34.858959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cf92a-e57d-4879-b189-936724796f62 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:34.859263 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cf92a-e57d-4879-b189-936724796f62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:34.867831 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cf92a-e57d-4879-b189-936724796f62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:34.873505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cf92a-e57d-4879-b189-936724796f62 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:34.878644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 692/1385] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:36.977211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753daf30-8107-4df2-87e1-c1eafce1ba5e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:36.977909 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753daf30-8107-4df2-87e1-c1eafce1ba5e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:36.977909 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753daf30-8107-4df2-87e1-c1eafce1ba5e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:36.979797 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 694/1386] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:36.985782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a79456-7d1d-46cf-b863-af5b8c1faba1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:36.986299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a79456-7d1d-46cf-b863-af5b8c1faba1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:36.986505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a79456-7d1d-46cf-b863-af5b8c1faba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:37.007925 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-62a79456-7d1d-46cf-b863-af5b8c1faba1 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:37.009282 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-62a79456-7d1d-46cf-b863-af5b8c1faba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:37.018870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 693/1387] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:37.068776 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753daf30-8107-4df2-87e1-c1eafce1ba5e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:37.083974 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753daf30-8107-4df2-87e1-c1eafce1ba5e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:37.092655 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fff94-2074-450a-93dc-85671a4473a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:37.093550 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fff94-2074-450a-93dc-85671a4473a2 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:37.093915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fff94-2074-450a-93dc-85671a4473a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:37.094278 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fff94-2074-450a-93dc-85671a4473a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:37.105892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fff94-2074-450a-93dc-85671a4473a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:37.107978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fff94-2074-450a-93dc-85671a4473a2 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:37.121029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 695/1388] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:39.375975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d48ff-841b-437c-8365-bb482889b79b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:39.376537 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d48ff-841b-437c-8365-bb482889b79b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:39.376643 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d48ff-841b-437c-8365-bb482889b79b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:39.378434 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 694/1389] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:39.383686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40f3fc-9c0f-4e9b-9fd3-d56f98dd52b0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:39.384132 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40f3fc-9c0f-4e9b-9fd3-d56f98dd52b0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:39.384278 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40f3fc-9c0f-4e9b-9fd3-d56f98dd52b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:39.402251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fc40f3fc-9c0f-4e9b-9fd3-d56f98dd52b0 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:39.403475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fc40f3fc-9c0f-4e9b-9fd3-d56f98dd52b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:39.411270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 696/1390] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:39.458732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d48ff-841b-437c-8365-bb482889b79b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:39.475718 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d48ff-841b-437c-8365-bb482889b79b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:39.481799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f6d93-7727-4e25-927e-7ede2d7a48b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:39.482446 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f6d93-7727-4e25-927e-7ede2d7a48b0 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:39.482758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f6d93-7727-4e25-927e-7ede2d7a48b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:39.483039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f6d93-7727-4e25-927e-7ede2d7a48b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:39.491252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f6d93-7727-4e25-927e-7ede2d7a48b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:39.492833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f6d93-7727-4e25-927e-7ede2d7a48b0 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:39.502710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 695/1391] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:42.243796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326078e-4eb3-42f9-b5fb-5ac97c56a883 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:42.243796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326078e-4eb3-42f9-b5fb-5ac97c56a883 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:42.244399 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326078e-4eb3-42f9-b5fb-5ac97c56a883 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:42.245442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 697/1392] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:42.250996 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df529de6-8eca-49c9-b091-ef9094069d35 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:42.251560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df529de6-8eca-49c9-b091-ef9094069d35 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:42.251667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df529de6-8eca-49c9-b091-ef9094069d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:42.269516 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-df529de6-8eca-49c9-b091-ef9094069d35 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:42.270727 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-df529de6-8eca-49c9-b091-ef9094069d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:42.279541 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 696/1393] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:11:42.324498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326078e-4eb3-42f9-b5fb-5ac97c56a883 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:42.340103 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326078e-4eb3-42f9-b5fb-5ac97c56a883 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:42.345561 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577216e5-477b-43f3-89e1-75769db2f88e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:42.346151 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577216e5-477b-43f3-89e1-75769db2f88e service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:42.346439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577216e5-477b-43f3-89e1-75769db2f88e service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:42.346720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577216e5-477b-43f3-89e1-75769db2f88e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:42.356197 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577216e5-477b-43f3-89e1-75769db2f88e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:42.357774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577216e5-477b-43f3-89e1-75769db2f88e service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:42.367447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 698/1394] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:44.204624 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f49eec-9e20-4a3c-834b-62042c169128 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:44.205179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f49eec-9e20-4a3c-834b-62042c169128 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:44.205179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f49eec-9e20-4a3c-834b-62042c169128 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:44.206551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 697/1395] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:44.211657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca91203-62fb-4c1d-adec-c17f3e2fcde9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:44.212080 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca91203-62fb-4c1d-adec-c17f3e2fcde9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:44.212218 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca91203-62fb-4c1d-adec-c17f3e2fcde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:44.231034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8ca91203-62fb-4c1d-adec-c17f3e2fcde9 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:44.232261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8ca91203-62fb-4c1d-adec-c17f3e2fcde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:44.240052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 699/1396] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:44.295981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36a176-d73a-4bd6-8b14-c1213131ddfd None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:44.296272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36a176-d73a-4bd6-8b14-c1213131ddfd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:44.296550 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36a176-d73a-4bd6-8b14-c1213131ddfd None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:44.297911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 698/1397] 10.222.0.67 () {54 vars in 962 bytes} [Fri Jun 24 15:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:44.302871 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc4371-8959-4f0c-ad34-c9ddd6ba58b4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:44.303162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc4371-8959-4f0c-ad34-c9ddd6ba58b4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:44.303478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc4371-8959-4f0c-ad34-c9ddd6ba58b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:44.352411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1bdc4371-8959-4f0c-ad34-c9ddd6ba58b4 None None] MFA Rules not processed for user `1971c464293f426f97d6cb199e48193b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:44.353568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1bdc4371-8959-4f0c-ad34-c9ddd6ba58b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:44.413141 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 700/1398] 10.222.0.67 () {58 vars in 1059 bytes} [Fri Jun 24 15:11:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:44.421007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36a176-d73a-4bd6-8b14-c1213131ddfd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:44.432077 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36a176-d73a-4bd6-8b14-c1213131ddfd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:44.435539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320208a0-33cf-4333-aa40-ba6c50f79301 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1971c464293f426f97d6cb199e48193b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:44.436172 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320208a0-33cf-4333-aa40-ba6c50f79301 service glance] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:44.436724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320208a0-33cf-4333-aa40-ba6c50f79301 service glance] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:44.437002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320208a0-33cf-4333-aa40-ba6c50f79301 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:44.446035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320208a0-33cf-4333-aa40-ba6c50f79301 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:44.448391 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320208a0-33cf-4333-aa40-ba6c50f79301 service glance] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:44.457657 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 699/1399] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:44.642664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744de02-9377-41cf-b1e8-fdbed493eb2f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:44.642953 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744de02-9377-41cf-b1e8-fdbed493eb2f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:44.643253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744de02-9377-41cf-b1e8-fdbed493eb2f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:44.644854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 701/1400] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:44.651328 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c62928-21ca-4ce7-8515-361fa57bd2ef None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:44.651632 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c62928-21ca-4ce7-8515-361fa57bd2ef None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:44.652000 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c62928-21ca-4ce7-8515-361fa57bd2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:44.671706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b7c62928-21ca-4ce7-8515-361fa57bd2ef None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:44.672960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b7c62928-21ca-4ce7-8515-361fa57bd2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:44.681293 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 700/1401] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:44.726416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744de02-9377-41cf-b1e8-fdbed493eb2f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:44.741663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744de02-9377-41cf-b1e8-fdbed493eb2f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:44.747245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6875826-4020-4fbe-a70a-1d47aac288a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:44.747980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6875826-4020-4fbe-a70a-1d47aac288a8 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:44.748202 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6875826-4020-4fbe-a70a-1d47aac288a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:44.748478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6875826-4020-4fbe-a70a-1d47aac288a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:44.756640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6875826-4020-4fbe-a70a-1d47aac288a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:44.758216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6875826-4020-4fbe-a70a-1d47aac288a8 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:44.768486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 702/1402] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:45.352390 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583e3ce-12e6-4bd6-b907-44e89d5e8fb3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:45.353232 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583e3ce-12e6-4bd6-b907-44e89d5e8fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:45.353554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583e3ce-12e6-4bd6-b907-44e89d5e8fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:45.402942 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4583e3ce-12e6-4bd6-b907-44e89d5e8fb3 None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:45.404208 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4583e3ce-12e6-4bd6-b907-44e89d5e8fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:45.470013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 701/1403] 10.222.0.67 () {58 vars in 1009 bytes} [Fri Jun 24 15:11:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:45.511844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5136ae-79e1-47fd-b8f8-77453660f0a3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:45.512356 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5136ae-79e1-47fd-b8f8-77453660f0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:45.512522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5136ae-79e1-47fd-b8f8-77453660f0a3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:45.514002 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 703/1404] 10.222.0.67 () {54 vars in 942 bytes} [Fri Jun 24 15:11:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:11:45.520593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a111d1-fe83-4ad5-a0c0-a80ab42958a6 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:45.520953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a111d1-fe83-4ad5-a0c0-a80ab42958a6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:45.521302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a111d1-fe83-4ad5-a0c0-a80ab42958a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:45.569260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b8a111d1-fe83-4ad5-a0c0-a80ab42958a6 None None] MFA Rules not processed for user `e4732ab3c8c446958092246d16b4b4f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:45.570450 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b8a111d1-fe83-4ad5-a0c0-a80ab42958a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:45.602816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:11:45.641235 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 702/1405] 10.222.0.67 () {58 vars in 1039 bytes} [Fri Jun 24 15:11:45 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:11:45.650944 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5136ae-79e1-47fd-b8f8-77453660f0a3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:45.658354 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:11:45.664388 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5136ae-79e1-47fd-b8f8-77453660f0a3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:45.670090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabd400-f0fd-4f21-ad71-c75fecfe489f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:45.670731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabd400-f0fd-4f21-ad71-c75fecfe489f service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:45.672772 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabd400-f0fd-4f21-ad71-c75fecfe489f service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:45.672772 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabd400-f0fd-4f21-ad71-c75fecfe489f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:45.679585 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdabd400-f0fd-4f21-ad71-c75fecfe489f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:45.681228 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdabd400-f0fd-4f21-ad71-c75fecfe489f service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:45.690657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 704/1406] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:11:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:11:47.083576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23ac79-43b1-4cf7-ae3d-ea105135a4ef None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:47.084290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23ac79-43b1-4cf7-ae3d-ea105135a4ef None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:47.084290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23ac79-43b1-4cf7-ae3d-ea105135a4ef None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:47.091979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16384ad-27b3-4fae-8e45-bb2013d39f3b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:47.092900 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16384ad-27b3-4fae-8e45-bb2013d39f3b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:47.093200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16384ad-27b3-4fae-8e45-bb2013d39f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:47.094822 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 703/1407] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:11:47 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:11:47.112591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e16384ad-27b3-4fae-8e45-bb2013d39f3b None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:11:47.115678 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e16384ad-27b3-4fae-8e45-bb2013d39f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:11:47.127515 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 705/1408] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:11:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:11:47.180984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23ac79-43b1-4cf7-ae3d-ea105135a4ef None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:11:47.197605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23ac79-43b1-4cf7-ae3d-ea105135a4ef None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:11:47.203194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db1747-ebd3-45a2-b8fe-120d42e8bdfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:11:47.203859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db1747-ebd3-45a2-b8fe-120d42e8bdfc service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:11:47.204148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db1747-ebd3-45a2-b8fe-120d42e8bdfc service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:11:47.204550 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db1747-ebd3-45a2-b8fe-120d42e8bdfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:11:47.213239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db1747-ebd3-45a2-b8fe-120d42e8bdfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:11:47.214959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db1747-ebd3-45a2-b8fe-120d42e8bdfc service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:11:47.226625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 704/1409] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:11:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:11:57.107800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=8 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:11:57.220741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=21 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:12:09.698872 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfe784-e27d-461a-93c7-c86952a59ead None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:09.699446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfe784-e27d-461a-93c7-c86952a59ead None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:09.699446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfe784-e27d-461a-93c7-c86952a59ead None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:09.700903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 706/1410] 10.222.0.67 () {54 vars in 918 bytes} [Fri Jun 24 15:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:09.705966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861350bc-99ed-44b1-94c7-6d4ac39c6600 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:09.706375 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861350bc-99ed-44b1-94c7-6d4ac39c6600 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:09.706535 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861350bc-99ed-44b1-94c7-6d4ac39c6600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:09.724651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-861350bc-99ed-44b1-94c7-6d4ac39c6600 None None] MFA Rules not processed for user `f612d6ff06fc458e8c1178cbf8321f79`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:09.725879 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-861350bc-99ed-44b1-94c7-6d4ac39c6600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:09.735100 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 705/1411] 10.222.0.67 () {58 vars in 1015 bytes} [Fri Jun 24 15:12:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:09.749687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de41a4-25ae-44fe-9a0b-6db548cf6fc6 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:09.749973 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de41a4-25ae-44fe-9a0b-6db548cf6fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:09.750282 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de41a4-25ae-44fe-9a0b-6db548cf6fc6 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:09.751703 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 707/1412] 10.222.0.67 () {54 vars in 918 bytes} [Fri Jun 24 15:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:09.756467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b27b7-c23d-4936-9e10-35526ba884bb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:09.756757 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b27b7-c23d-4936-9e10-35526ba884bb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:09.757034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b27b7-c23d-4936-9e10-35526ba884bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:09.775538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f32b27b7-c23d-4936-9e10-35526ba884bb None None] MFA Rules not processed for user `51de59caa3144a1cbf12b20b15e2134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:09.776716 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f32b27b7-c23d-4936-9e10-35526ba884bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:09.785818 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 706/1413] 10.222.0.67 () {58 vars in 1015 bytes} [Fri Jun 24 15:12:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:12.272223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1b1d2-e34d-40ec-9928-1108ded44042 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:12.272665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1b1d2-e34d-40ec-9928-1108ded44042 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:12.272768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1b1d2-e34d-40ec-9928-1108ded44042 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:12.274656 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 15:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:12.279774 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3df0e1-3a39-46e7-9504-eeaab2faa21b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:12.280214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3df0e1-3a39-46e7-9504-eeaab2faa21b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:12.280314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3df0e1-3a39-46e7-9504-eeaab2faa21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:12.298859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ff3df0e1-3a39-46e7-9504-eeaab2faa21b None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:12.306240 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ff3df0e1-3a39-46e7-9504-eeaab2faa21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:12.308051 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 15:12:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:12.353385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1b1d2-e34d-40ec-9928-1108ded44042 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:12.365660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1b1d2-e34d-40ec-9928-1108ded44042 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:12.371405 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464481be-6269-4366-9767-10c8a74a700a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:12.372047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464481be-6269-4366-9767-10c8a74a700a service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:12.372332 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464481be-6269-4366-9767-10c8a74a700a service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:12.372612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464481be-6269-4366-9767-10c8a74a700a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:12.380848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464481be-6269-4366-9767-10c8a74a700a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:12.382436 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464481be-6269-4366-9767-10c8a74a700a service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:12.392020 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 709/1416] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:13.282991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f067700-a57b-4f55-a315-999984275c59 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:13.283581 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f067700-a57b-4f55-a315-999984275c59 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:13.283581 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f067700-a57b-4f55-a315-999984275c59 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:13.284942 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 15:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:13.290803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca69b98-ae94-4121-bf92-32291245d0c6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:13.291237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca69b98-ae94-4121-bf92-32291245d0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:13.291455 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca69b98-ae94-4121-bf92-32291245d0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:13.309406 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-cca69b98-ae94-4121-bf92-32291245d0c6 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:13.310590 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cca69b98-ae94-4121-bf92-32291245d0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:13.318305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 15:12:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:13.466348 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f067700-a57b-4f55-a315-999984275c59 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:13.480751 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f067700-a57b-4f55-a315-999984275c59 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:13.486076 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43d015-71b7-4478-846b-32fc03bb45ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:13.486710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43d015-71b7-4478-846b-32fc03bb45ce service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:13.486990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43d015-71b7-4478-846b-32fc03bb45ce service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:13.487308 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43d015-71b7-4478-846b-32fc03bb45ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:13.495895 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43d015-71b7-4478-846b-32fc03bb45ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:13.497470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43d015-71b7-4478-846b-32fc03bb45ce service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:13.506665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 709/1419] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:14.396646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f5ed2-49e2-487c-9c10-7024b109ca67 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:14.397187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f5ed2-49e2-487c-9c10-7024b109ca67 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:14.397187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f5ed2-49e2-487c-9c10-7024b109ca67 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:14.398591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 15:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:14.403841 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224dc06-8b4f-45b6-89f4-712e808ce621 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:14.404257 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224dc06-8b4f-45b6-89f4-712e808ce621 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:14.404433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224dc06-8b4f-45b6-89f4-712e808ce621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:14.422354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e224dc06-8b4f-45b6-89f4-712e808ce621 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:14.423588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e224dc06-8b4f-45b6-89f4-712e808ce621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:14.431915 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 15:12:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:14.585097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f5ed2-49e2-487c-9c10-7024b109ca67 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:14.600688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f5ed2-49e2-487c-9c10-7024b109ca67 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:14.607243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9f5ef-e787-435e-9b52-693fea7f11d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:14.607891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9f5ef-e787-435e-9b52-693fea7f11d9 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:14.608175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9f5ef-e787-435e-9b52-693fea7f11d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:14.608450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9f5ef-e787-435e-9b52-693fea7f11d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:14.616975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d9f5ef-e787-435e-9b52-693fea7f11d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:14.618667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d9f5ef-e787-435e-9b52-693fea7f11d9 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:14.628121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 712/1422] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:15.488806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21108b8-980f-43dd-ad5f-336b88452b61 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:15.489383 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21108b8-980f-43dd-ad5f-336b88452b61 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:15.489383 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21108b8-980f-43dd-ad5f-336b88452b61 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:15.490779 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 15:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:15.496237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf7689-ea5e-40fd-850b-3e302874b649 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:15.496642 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf7689-ea5e-40fd-850b-3e302874b649 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:15.496834 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf7689-ea5e-40fd-850b-3e302874b649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:15.514989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0faf7689-ea5e-40fd-850b-3e302874b649 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:15.516204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0faf7689-ea5e-40fd-850b-3e302874b649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:15.523902 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 15:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:15.673897 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21108b8-980f-43dd-ad5f-336b88452b61 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:15.688831 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21108b8-980f-43dd-ad5f-336b88452b61 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:15.694217 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ed9f2-e03c-4250-b6b7-25cc205a7817 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:15.694858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ed9f2-e03c-4250-b6b7-25cc205a7817 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:15.695138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ed9f2-e03c-4250-b6b7-25cc205a7817 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:15.695447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ed9f2-e03c-4250-b6b7-25cc205a7817 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:15.703713 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ed9f2-e03c-4250-b6b7-25cc205a7817 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:15.705278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ed9f2-e03c-4250-b6b7-25cc205a7817 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:15.714785 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 712/1425] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:16.595455 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02093d4d-d291-444a-8abf-fd0725157602 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:16.596044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02093d4d-d291-444a-8abf-fd0725157602 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:16.596044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02093d4d-d291-444a-8abf-fd0725157602 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:16.597376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 15:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:16.603002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5de99-e12e-4ef8-a21d-eac1551074a5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:16.603430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5de99-e12e-4ef8-a21d-eac1551074a5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:16.603635 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5de99-e12e-4ef8-a21d-eac1551074a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:16.622240 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-34a5de99-e12e-4ef8-a21d-eac1551074a5 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:16.623447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-34a5de99-e12e-4ef8-a21d-eac1551074a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:16.631067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 15:12:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:16.780253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02093d4d-d291-444a-8abf-fd0725157602 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:16.795550 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02093d4d-d291-444a-8abf-fd0725157602 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:16.800914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d60558-4720-4b42-9ad9-d3fd134bf7c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:16.801551 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d60558-4720-4b42-9ad9-d3fd134bf7c0 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:16.801834 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d60558-4720-4b42-9ad9-d3fd134bf7c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:16.802106 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d60558-4720-4b42-9ad9-d3fd134bf7c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:16.810365 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d60558-4720-4b42-9ad9-d3fd134bf7c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:16.811976 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d60558-4720-4b42-9ad9-d3fd134bf7c0 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:16.821320 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 715/1428] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:17.659209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f0c10-1f02-4ac8-bd46-1e64c0cba547 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:17.659793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f0c10-1f02-4ac8-bd46-1e64c0cba547 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:17.659793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f0c10-1f02-4ac8-bd46-1e64c0cba547 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:17.661150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 15:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:17.666830 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf881e2-65c6-41a0-bacf-fe9a39d4ff7b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:17.667249 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf881e2-65c6-41a0-bacf-fe9a39d4ff7b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:17.667440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf881e2-65c6-41a0-bacf-fe9a39d4ff7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:17.686645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fbf881e2-65c6-41a0-bacf-fe9a39d4ff7b None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:17.687855 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fbf881e2-65c6-41a0-bacf-fe9a39d4ff7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:17.695969 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:17.846991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f0c10-1f02-4ac8-bd46-1e64c0cba547 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:17.861231 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f0c10-1f02-4ac8-bd46-1e64c0cba547 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:17.866482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea3006-885c-43b4-937c-14caa023451c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:17.867111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea3006-885c-43b4-937c-14caa023451c service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:17.867422 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea3006-885c-43b4-937c-14caa023451c service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:17.867749 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea3006-885c-43b4-937c-14caa023451c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:17.875796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea3006-885c-43b4-937c-14caa023451c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:17.877370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea3006-885c-43b4-937c-14caa023451c service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:17.886962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 715/1431] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:18.733120 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518c95c-be17-442f-a947-79bc97a42f7f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:18.733766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518c95c-be17-442f-a947-79bc97a42f7f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:18.733766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518c95c-be17-442f-a947-79bc97a42f7f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:18.735047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 15:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:18.740221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4670cf-3007-4f3c-818b-a6fac20cf186 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:18.740650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4670cf-3007-4f3c-818b-a6fac20cf186 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:18.740781 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4670cf-3007-4f3c-818b-a6fac20cf186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:18.759633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9b4670cf-3007-4f3c-818b-a6fac20cf186 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:18.760824 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9b4670cf-3007-4f3c-818b-a6fac20cf186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:18.768761 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 15:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:18.920991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518c95c-be17-442f-a947-79bc97a42f7f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:18.935223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518c95c-be17-442f-a947-79bc97a42f7f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:18.940578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99244663-9590-4aa5-b371-de954178dfaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:18.941215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99244663-9590-4aa5-b371-de954178dfaa service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:18.941503 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99244663-9590-4aa5-b371-de954178dfaa service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:18.941800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99244663-9590-4aa5-b371-de954178dfaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:18.950134 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99244663-9590-4aa5-b371-de954178dfaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:18.951757 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99244663-9590-4aa5-b371-de954178dfaa service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:18.961034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 718/1434] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:19.841622 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb943f7-df9d-4ca3-bfb2-77415bae739e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:19.842185 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb943f7-df9d-4ca3-bfb2-77415bae739e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:19.842185 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb943f7-df9d-4ca3-bfb2-77415bae739e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:19.843598 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 15:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:19.848425 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ac1d7-d17a-4726-bac9-d7588ccbf71c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:19.848875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ac1d7-d17a-4726-bac9-d7588ccbf71c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:19.848995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ac1d7-d17a-4726-bac9-d7588ccbf71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:19.866693 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9a1ac1d7-d17a-4726-bac9-d7588ccbf71c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:19.868393 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9a1ac1d7-d17a-4726-bac9-d7588ccbf71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:19.876203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 15:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:20.026848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb943f7-df9d-4ca3-bfb2-77415bae739e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:20.041829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb943f7-df9d-4ca3-bfb2-77415bae739e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:20.047131 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66da33-9379-49f3-9432-4ddb8222da32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:20.047803 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66da33-9379-49f3-9432-4ddb8222da32 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:20.048088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66da33-9379-49f3-9432-4ddb8222da32 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:20.048366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66da33-9379-49f3-9432-4ddb8222da32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:20.056993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66da33-9379-49f3-9432-4ddb8222da32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:20.058551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66da33-9379-49f3-9432-4ddb8222da32 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:20.067887 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 718/1437] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:20.916758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a2de6-09bb-4bd3-ac7a-f79d1543462c None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:20.917312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a2de6-09bb-4bd3-ac7a-f79d1543462c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:20.917312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a2de6-09bb-4bd3-ac7a-f79d1543462c None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:20.918739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 15:12:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:20.923710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7da9c-c6c6-48d1-ae3b-c485cd5a9073 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:20.924114 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7da9c-c6c6-48d1-ae3b-c485cd5a9073 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:20.924291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7da9c-c6c6-48d1-ae3b-c485cd5a9073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:20.942069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-aac7da9c-c6c6-48d1-ae3b-c485cd5a9073 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:20.943287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-aac7da9c-c6c6-48d1-ae3b-c485cd5a9073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:20.952043 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 15:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:21.101474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a2de6-09bb-4bd3-ac7a-f79d1543462c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:21.115606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a2de6-09bb-4bd3-ac7a-f79d1543462c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:21.122012 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058084d9-e584-46ed-a08a-4c1c459b4900 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:21.122654 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058084d9-e584-46ed-a08a-4c1c459b4900 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:21.122944 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058084d9-e584-46ed-a08a-4c1c459b4900 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:21.123258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058084d9-e584-46ed-a08a-4c1c459b4900 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:21.131487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058084d9-e584-46ed-a08a-4c1c459b4900 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:21.133067 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058084d9-e584-46ed-a08a-4c1c459b4900 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:21.143368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 721/1440] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:23.092138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c1da9-07d9-4317-8470-e3b705c70e79 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:23.092656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c1da9-07d9-4317-8470-e3b705c70e79 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:23.092753 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c1da9-07d9-4317-8470-e3b705c70e79 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:23.094227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 15:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:23.099508 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef47de-b9bf-45d2-9e68-1cbff8ed117b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:23.099962 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef47de-b9bf-45d2-9e68-1cbff8ed117b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:23.100071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef47de-b9bf-45d2-9e68-1cbff8ed117b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:23.118691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b2ef47de-b9bf-45d2-9e68-1cbff8ed117b None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:23.119907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b2ef47de-b9bf-45d2-9e68-1cbff8ed117b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:23.127546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 15:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:23.166596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c1da9-07d9-4317-8470-e3b705c70e79 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:23.177241 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c1da9-07d9-4317-8470-e3b705c70e79 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:23.181391 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18754c24-0e8d-4ed9-8cd4-5c276ebb4d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:23.182013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18754c24-0e8d-4ed9-8cd4-5c276ebb4d4e service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:23.182579 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18754c24-0e8d-4ed9-8cd4-5c276ebb4d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:23.182858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18754c24-0e8d-4ed9-8cd4-5c276ebb4d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:23.191665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18754c24-0e8d-4ed9-8cd4-5c276ebb4d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:23.193075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18754c24-0e8d-4ed9-8cd4-5c276ebb4d4e service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:23.203709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 721/1443] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:25.369022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35a450-90f0-4b84-9160-ffa0cf8925fd None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:25.369552 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35a450-90f0-4b84-9160-ffa0cf8925fd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:25.369552 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35a450-90f0-4b84-9160-ffa0cf8925fd None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:25.370886 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 15:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:25.376219 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7b3b2-d70c-4970-aeea-2114483dc195 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:25.376671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7b3b2-d70c-4970-aeea-2114483dc195 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:25.376776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7b3b2-d70c-4970-aeea-2114483dc195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:25.394494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5cc7b3b2-d70c-4970-aeea-2114483dc195 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:25.395700 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5cc7b3b2-d70c-4970-aeea-2114483dc195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:25.403951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 15:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:25.442005 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35a450-90f0-4b84-9160-ffa0cf8925fd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:25.453466 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35a450-90f0-4b84-9160-ffa0cf8925fd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:25.456975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9d3fa-78b4-4f81-a071-b83d153ec0b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:25.457607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9d3fa-78b4-4f81-a071-b83d153ec0b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:25.457893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9d3fa-78b4-4f81-a071-b83d153ec0b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:25.458173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9d3fa-78b4-4f81-a071-b83d153ec0b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:25.467595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9d3fa-78b4-4f81-a071-b83d153ec0b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:25.469051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9d3fa-78b4-4f81-a071-b83d153ec0b7 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:25.478272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 724/1446] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:27.853484 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6b47d-1e01-4255-9c12-bf7c6c75191b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:27.854075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6b47d-1e01-4255-9c12-bf7c6c75191b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:27.854075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6b47d-1e01-4255-9c12-bf7c6c75191b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:27.855469 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 15:12:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:27.860905 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18accb5c-58ff-407f-aac6-713b340dfcd8 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:27.861335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18accb5c-58ff-407f-aac6-713b340dfcd8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:27.861443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18accb5c-58ff-407f-aac6-713b340dfcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:27.879161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-18accb5c-58ff-407f-aac6-713b340dfcd8 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:27.880413 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-18accb5c-58ff-407f-aac6-713b340dfcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:27.888323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 15:12:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:27.927563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6b47d-1e01-4255-9c12-bf7c6c75191b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:27.938732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6b47d-1e01-4255-9c12-bf7c6c75191b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:27.942225 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf7b67-bafb-4612-8e6f-1409cd75ad90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:27.942856 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf7b67-bafb-4612-8e6f-1409cd75ad90 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:27.943134 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf7b67-bafb-4612-8e6f-1409cd75ad90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:27.943447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf7b67-bafb-4612-8e6f-1409cd75ad90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:27.952559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf7b67-bafb-4612-8e6f-1409cd75ad90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:27.953969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf7b67-bafb-4612-8e6f-1409cd75ad90 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:27.963652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 724/1449] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:30.145352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fd024-0ec9-4f6f-9488-4abe51cc2fc7 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:30.145903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fd024-0ec9-4f6f-9488-4abe51cc2fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:30.145903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fd024-0ec9-4f6f-9488-4abe51cc2fc7 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:30.147356 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 15:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:30.153247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d4d8c-d15f-4e8a-a05e-05b97b99f7f3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:30.153690 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d4d8c-d15f-4e8a-a05e-05b97b99f7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:30.153809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d4d8c-d15f-4e8a-a05e-05b97b99f7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:30.172169 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a40d4d8c-d15f-4e8a-a05e-05b97b99f7f3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:30.173354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a40d4d8c-d15f-4e8a-a05e-05b97b99f7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:30.181090 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:30.220000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fd024-0ec9-4f6f-9488-4abe51cc2fc7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:30.231261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fd024-0ec9-4f6f-9488-4abe51cc2fc7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:30.234736 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc0a3f-2d50-4874-8f6c-5452df6e9454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:30.235399 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc0a3f-2d50-4874-8f6c-5452df6e9454 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:30.235682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc0a3f-2d50-4874-8f6c-5452df6e9454 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:30.235958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc0a3f-2d50-4874-8f6c-5452df6e9454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:30.244731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc0a3f-2d50-4874-8f6c-5452df6e9454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:30.246149 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc0a3f-2d50-4874-8f6c-5452df6e9454 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:30.255369 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 727/1452] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:32.341496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41861ccc-0f0e-4439-be66-879570e74ccd None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:32.342024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41861ccc-0f0e-4439-be66-879570e74ccd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:32.342118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41861ccc-0f0e-4439-be66-879570e74ccd None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:32.343797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 15:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:32.349146 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcb6b0-ffb1-4b99-ab0c-53674ef3fd60 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:32.349586 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcb6b0-ffb1-4b99-ab0c-53674ef3fd60 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:32.349701 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcb6b0-ffb1-4b99-ab0c-53674ef3fd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:32.366863 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fbdcb6b0-ffb1-4b99-ab0c-53674ef3fd60 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:32.368606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fbdcb6b0-ffb1-4b99-ab0c-53674ef3fd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:32.376251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 15:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:32.413921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41861ccc-0f0e-4439-be66-879570e74ccd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:32.424930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41861ccc-0f0e-4439-be66-879570e74ccd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:32.428458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e0c3b-9ce3-4c2e-978e-1929f6481ffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:32.429096 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e0c3b-9ce3-4c2e-978e-1929f6481ffa service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:32.429388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e0c3b-9ce3-4c2e-978e-1929f6481ffa service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:32.430209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e0c3b-9ce3-4c2e-978e-1929f6481ffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:32.439012 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e0c3b-9ce3-4c2e-978e-1929f6481ffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:32.440471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e0c3b-9ce3-4c2e-978e-1929f6481ffa service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:32.450306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 727/1455] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:34.539849 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb6df0-fdb4-42b2-97fa-d0421876275e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:34.540388 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb6df0-fdb4-42b2-97fa-d0421876275e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:34.540665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb6df0-fdb4-42b2-97fa-d0421876275e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:34.542065 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 15:12:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:34.547372 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bf1a9-e6dc-43ac-82ff-31b6bb6f6501 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:34.547860 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bf1a9-e6dc-43ac-82ff-31b6bb6f6501 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:34.547970 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bf1a9-e6dc-43ac-82ff-31b6bb6f6501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:34.565747 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-853bf1a9-e6dc-43ac-82ff-31b6bb6f6501 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:34.566951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-853bf1a9-e6dc-43ac-82ff-31b6bb6f6501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:34.574930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 15:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:34.612440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb6df0-fdb4-42b2-97fa-d0421876275e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:34.623996 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb6df0-fdb4-42b2-97fa-d0421876275e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:34.627617 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755415c-43bf-44d1-a864-59e05157a9b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:34.628305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755415c-43bf-44d1-a864-59e05157a9b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:34.628594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755415c-43bf-44d1-a864-59e05157a9b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:34.628876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755415c-43bf-44d1-a864-59e05157a9b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:34.637898 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755415c-43bf-44d1-a864-59e05157a9b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:34.639404 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755415c-43bf-44d1-a864-59e05157a9b4 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:34.649485 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 730/1458] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:36.751106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc760a-72f8-470f-831e-f803f4da1cb1 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:36.751679 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc760a-72f8-470f-831e-f803f4da1cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:36.751679 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc760a-72f8-470f-831e-f803f4da1cb1 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:36.753097 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 15:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:36.758646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161eb4fe-48a2-40ed-897f-21182ed9a45f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:36.759082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161eb4fe-48a2-40ed-897f-21182ed9a45f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:36.759270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161eb4fe-48a2-40ed-897f-21182ed9a45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:36.777205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-161eb4fe-48a2-40ed-897f-21182ed9a45f None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:36.778384 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-161eb4fe-48a2-40ed-897f-21182ed9a45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:36.786077 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 15:12:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:36.824632 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc760a-72f8-470f-831e-f803f4da1cb1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:36.835112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc760a-72f8-470f-831e-f803f4da1cb1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:36.839045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2d3c5-ba3a-4bac-a5f8-24cbdd35a2f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:36.839713 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2d3c5-ba3a-4bac-a5f8-24cbdd35a2f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:36.840000 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2d3c5-ba3a-4bac-a5f8-24cbdd35a2f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:36.840276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2d3c5-ba3a-4bac-a5f8-24cbdd35a2f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:36.848975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2d3c5-ba3a-4bac-a5f8-24cbdd35a2f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:36.850395 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2d3c5-ba3a-4bac-a5f8-24cbdd35a2f1 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:36.860210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 730/1461] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:38.968378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188bda9-b9e8-4c58-8098-dc1af49966de None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:38.968977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188bda9-b9e8-4c58-8098-dc1af49966de None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:38.968977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188bda9-b9e8-4c58-8098-dc1af49966de None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:38.970296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 15:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:38.975402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d6904-fd1f-438a-b9d7-eaed6f553f00 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:38.975816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d6904-fd1f-438a-b9d7-eaed6f553f00 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:38.975965 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d6904-fd1f-438a-b9d7-eaed6f553f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:38.982476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=79 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:12:38.994724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8e1d6904-fd1f-438a-b9d7-eaed6f553f00 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:38.995933 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8e1d6904-fd1f-438a-b9d7-eaed6f553f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:39.003633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 15:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:39.041236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188bda9-b9e8-4c58-8098-dc1af49966de None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:39.046570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=73 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:12:39.052519 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188bda9-b9e8-4c58-8098-dc1af49966de None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:39.057242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd59075-7e36-468d-a9fd-535f92da2dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:39.068951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd59075-7e36-468d-a9fd-535f92da2dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:39.068951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd59075-7e36-468d-a9fd-535f92da2dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:39.068951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd59075-7e36-468d-a9fd-535f92da2dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:39.068951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd59075-7e36-468d-a9fd-535f92da2dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:39.069938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd59075-7e36-468d-a9fd-535f92da2dd3 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:39.080110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 733/1464] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:41.144794 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cee8b-84e2-47c6-a805-213e582e6c95 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:41.145306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cee8b-84e2-47c6-a805-213e582e6c95 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:41.145416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cee8b-84e2-47c6-a805-213e582e6c95 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:41.146745 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 15:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:41.152283 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f835a00-e8b6-4989-ad2f-adeb0dbcda27 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:41.152701 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f835a00-e8b6-4989-ad2f-adeb0dbcda27 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:41.152832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f835a00-e8b6-4989-ad2f-adeb0dbcda27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:41.170693 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2f835a00-e8b6-4989-ad2f-adeb0dbcda27 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:41.171911 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2f835a00-e8b6-4989-ad2f-adeb0dbcda27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:41.180132 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 15:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:41.215736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cee8b-84e2-47c6-a805-213e582e6c95 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:41.226996 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cee8b-84e2-47c6-a805-213e582e6c95 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:41.231001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf29694-1cd4-40a3-bd13-060c41bf146a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:41.231693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf29694-1cd4-40a3-bd13-060c41bf146a service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:41.231976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf29694-1cd4-40a3-bd13-060c41bf146a service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:41.232251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf29694-1cd4-40a3-bd13-060c41bf146a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:41.241447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf29694-1cd4-40a3-bd13-060c41bf146a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:41.243796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf29694-1cd4-40a3-bd13-060c41bf146a service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:41.253160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 733/1467] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:43.337345 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07363594-9464-40d1-bb68-c90d49339e62 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:43.337895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07363594-9464-40d1-bb68-c90d49339e62 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:43.338019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07363594-9464-40d1-bb68-c90d49339e62 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:43.339330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 15:12:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:12:43.344625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad10b4e-ef04-4e17-bc59-9b1eb011d5c1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:43.345037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad10b4e-ef04-4e17-bc59-9b1eb011d5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:43.345174 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad10b4e-ef04-4e17-bc59-9b1eb011d5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:43.362987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3ad10b4e-ef04-4e17-bc59-9b1eb011d5c1 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:43.364132 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3ad10b4e-ef04-4e17-bc59-9b1eb011d5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:43.371844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 15:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:12:43.408931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07363594-9464-40d1-bb68-c90d49339e62 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:43.419738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07363594-9464-40d1-bb68-c90d49339e62 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:43.423255 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b642622-0c2e-43cd-9c85-8318dfd46be2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:43.423887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b642622-0c2e-43cd-9c85-8318dfd46be2 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:43.424171 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b642622-0c2e-43cd-9c85-8318dfd46be2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:43.424447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b642622-0c2e-43cd-9c85-8318dfd46be2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:43.433126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b642622-0c2e-43cd-9c85-8318dfd46be2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:43.434546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b642622-0c2e-43cd-9c85-8318dfd46be2 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:43.443776 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 736/1470] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:53.357932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=11 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:12:53.439657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=14 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:13:03.662890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc7d27-665b-4c95-aefc-ba455a814437 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:03.663403 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc7d27-665b-4c95-aefc-ba455a814437 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:03.663507 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc7d27-665b-4c95-aefc-ba455a814437 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:03.665248 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 735/1471] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:13:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:13:03.670810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b258f93-b5ed-4bb6-8cc6-6e9c93517b27 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:03.671243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b258f93-b5ed-4bb6-8cc6-6e9c93517b27 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:03.671401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b258f93-b5ed-4bb6-8cc6-6e9c93517b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:03.688840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7b258f93-b5ed-4bb6-8cc6-6e9c93517b27 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:13:03.690066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7b258f93-b5ed-4bb6-8cc6-6e9c93517b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:13:03.698556 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 737/1472] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:13:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:13:03.747488 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f2b7e-0531-4288-9df2-3f71b27468d6 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:03.747777 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f2b7e-0531-4288-9df2-3f71b27468d6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:03.748063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f2b7e-0531-4288-9df2-3f71b27468d6 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:03.749433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 736/1473] 10.222.0.67 () {54 vars in 962 bytes} [Fri Jun 24 15:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:13:03.754350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849bebc-9b5f-4f5f-bc42-f3343eba27f3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:03.754641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849bebc-9b5f-4f5f-bc42-f3343eba27f3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:03.754947 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849bebc-9b5f-4f5f-bc42-f3343eba27f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:03.774024 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e849bebc-9b5f-4f5f-bc42-f3343eba27f3 None None] MFA Rules not processed for user `1971c464293f426f97d6cb199e48193b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:13:03.775244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e849bebc-9b5f-4f5f-bc42-f3343eba27f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:13:03.781729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 738/1474] 10.222.0.67 () {58 vars in 1059 bytes} [Fri Jun 24 15:13:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:13:03.789284 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f2b7e-0531-4288-9df2-3f71b27468d6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:13:03.799776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f2b7e-0531-4288-9df2-3f71b27468d6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:13:03.803285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103b75c-2075-4514-b4d6-efac0906c621 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1971c464293f426f97d6cb199e48193b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:13:03.803928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103b75c-2075-4514-b4d6-efac0906c621 service glance] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:03.804209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103b75c-2075-4514-b4d6-efac0906c621 service glance] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:03.804479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103b75c-2075-4514-b4d6-efac0906c621 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:03.813105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103b75c-2075-4514-b4d6-efac0906c621 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:13:03.814511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103b75c-2075-4514-b4d6-efac0906c621 service glance] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:13:03.824258 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 737/1475] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:13:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:13:05.986585 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686aba9-4c97-483c-b141-b9a414fa622a None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:05.987169 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686aba9-4c97-483c-b141-b9a414fa622a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:05.987169 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686aba9-4c97-483c-b141-b9a414fa622a None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:05.988552 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 739/1476] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:13:05.994580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f2ab3-b1f0-4826-9c4e-ad07d6d4ce22 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:05.994975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f2ab3-b1f0-4826-9c4e-ad07d6d4ce22 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:05.996767 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f2ab3-b1f0-4826-9c4e-ad07d6d4ce22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:06.012984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4a9f2ab3-b1f0-4826-9c4e-ad07d6d4ce22 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:13:06.014219 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4a9f2ab3-b1f0-4826-9c4e-ad07d6d4ce22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:13:06.022171 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 738/1477] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:13:06.046126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686aba9-4c97-483c-b141-b9a414fa622a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:13:06.059749 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686aba9-4c97-483c-b141-b9a414fa622a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:13:06.064289 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439e1ba-82c8-46bc-9fb1-796dbfa1c456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:13:06.065549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439e1ba-82c8-46bc-9fb1-796dbfa1c456 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:06.065549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439e1ba-82c8-46bc-9fb1-796dbfa1c456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:06.065549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439e1ba-82c8-46bc-9fb1-796dbfa1c456 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:06.066411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439e1ba-82c8-46bc-9fb1-796dbfa1c456 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:13:06.068211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-4439e1ba-82c8-46bc-9fb1-796dbfa1c456 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 15:13:06.068211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jun 24 15:13:06.068211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: Jun 24 15:13:06.069356 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439e1ba-82c8-46bc-9fb1-796dbfa1c456 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:13:06.088562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 740/1478] 10.222.0.67 () {56 vars in 1108 bytes} [Fri Jun 24 15:13:06 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:13:08.134408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da64cfd-6d8d-4e61-8aeb-4cb7a2f750de None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:08.134909 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da64cfd-6d8d-4e61-8aeb-4cb7a2f750de None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:08.135006 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da64cfd-6d8d-4e61-8aeb-4cb7a2f750de None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:08.136483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 739/1479] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:13:08.141860 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56255096-f656-4d31-b5a7-c8be207c0b47 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:08.142271 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56255096-f656-4d31-b5a7-c8be207c0b47 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:08.142404 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56255096-f656-4d31-b5a7-c8be207c0b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:08.160490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-56255096-f656-4d31-b5a7-c8be207c0b47 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:13:08.161704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-56255096-f656-4d31-b5a7-c8be207c0b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:13:08.169737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 741/1480] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:13:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:13:08.213524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da64cfd-6d8d-4e61-8aeb-4cb7a2f750de None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:13:08.227581 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da64cfd-6d8d-4e61-8aeb-4cb7a2f750de None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:13:08.232841 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b01e1-3823-4b89-8136-bd50d1a31970 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:13:08.233470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b01e1-3823-4b89-8136-bd50d1a31970 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:08.233747 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b01e1-3823-4b89-8136-bd50d1a31970 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:08.234021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b01e1-3823-4b89-8136-bd50d1a31970 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:08.242070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b01e1-3823-4b89-8136-bd50d1a31970 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:13:08.243671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b01e1-3823-4b89-8136-bd50d1a31970 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:13:08.252819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 740/1481] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:13:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:13:10.456762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dec1cb-376d-4ba8-b9b3-0dafafb8b98a None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:10.457343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dec1cb-376d-4ba8-b9b3-0dafafb8b98a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:10.457343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dec1cb-376d-4ba8-b9b3-0dafafb8b98a None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:10.458701 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 742/1482] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:13:10.464480 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610519cd-c3bb-42e4-8150-d1ca37af2d0e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:10.464898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610519cd-c3bb-42e4-8150-d1ca37af2d0e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:10.470678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610519cd-c3bb-42e4-8150-d1ca37af2d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:10.484495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-610519cd-c3bb-42e4-8150-d1ca37af2d0e None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:13:10.485703 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-610519cd-c3bb-42e4-8150-d1ca37af2d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:13:10.494616 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 741/1483] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:13:10.531892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dec1cb-376d-4ba8-b9b3-0dafafb8b98a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:13:10.543633 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dec1cb-376d-4ba8-b9b3-0dafafb8b98a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:13:10.547127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c6c23-3973-4b86-b005-8919c407be7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:13:10.547794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c6c23-3973-4b86-b005-8919c407be7f service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:10.548081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c6c23-3973-4b86-b005-8919c407be7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:10.548358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c6c23-3973-4b86-b005-8919c407be7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:10.558244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c6c23-3973-4b86-b005-8919c407be7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:13:10.559743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c6c23-3973-4b86-b005-8919c407be7f service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:13:10.569297 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 743/1484] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:13:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:13:12.660278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902f22b-e6ce-466d-954e-0c3bdab084d8 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:12.660820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902f22b-e6ce-466d-954e-0c3bdab084d8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:12.660820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902f22b-e6ce-466d-954e-0c3bdab084d8 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:12.662207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 742/1485] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:13:12.667684 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa99f06-16af-4591-86de-8ce00d9b0ed6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:12.668081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa99f06-16af-4591-86de-8ce00d9b0ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:12.668277 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa99f06-16af-4591-86de-8ce00d9b0ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:12.685935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0fa99f06-16af-4591-86de-8ce00d9b0ed6 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:13:12.687130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0fa99f06-16af-4591-86de-8ce00d9b0ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:13:12.695600 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 744/1486] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:13:12.734096 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902f22b-e6ce-466d-954e-0c3bdab084d8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:13:12.746475 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902f22b-e6ce-466d-954e-0c3bdab084d8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:13:12.748886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa3827-0702-4a93-a4e7-9e513edc1257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:13:12.749528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa3827-0702-4a93-a4e7-9e513edc1257 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:12.749812 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa3827-0702-4a93-a4e7-9e513edc1257 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:12.750088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa3827-0702-4a93-a4e7-9e513edc1257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:12.760003 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa3827-0702-4a93-a4e7-9e513edc1257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:13:12.761469 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa3827-0702-4a93-a4e7-9e513edc1257 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:13:12.770863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 743/1487] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:13:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:13:14.904835 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fecedd-4f5e-4673-97dc-b5a1fb87e499 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:14.905398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fecedd-4f5e-4673-97dc-b5a1fb87e499 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:14.905559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fecedd-4f5e-4673-97dc-b5a1fb87e499 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:14.907157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 745/1488] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:13:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:13:14.912635 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a152f7d-1092-477f-93be-4d168160bd44 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:14.913120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a152f7d-1092-477f-93be-4d168160bd44 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:14.913293 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a152f7d-1092-477f-93be-4d168160bd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:14.930845 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0a152f7d-1092-477f-93be-4d168160bd44 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:13:14.932062 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0a152f7d-1092-477f-93be-4d168160bd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:13:14.939867 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 744/1489] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:13:14.980279 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fecedd-4f5e-4673-97dc-b5a1fb87e499 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:13:14.991263 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fecedd-4f5e-4673-97dc-b5a1fb87e499 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:13:14.994826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca49ac-31c9-4bf4-960a-766c4ea6a826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:13:14.995497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca49ac-31c9-4bf4-960a-766c4ea6a826 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:14.995784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca49ac-31c9-4bf4-960a-766c4ea6a826 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:14.996064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca49ac-31c9-4bf4-960a-766c4ea6a826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:15.004839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca49ac-31c9-4bf4-960a-766c4ea6a826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:13:15.006815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca49ac-31c9-4bf4-960a-766c4ea6a826 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:13:15.016355 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 746/1490] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:13:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:13:17.373745 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b32c1-ae1d-4651-8740-1b3fabdcecbc None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:17.374306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b32c1-ae1d-4651-8740-1b3fabdcecbc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:17.374306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b32c1-ae1d-4651-8740-1b3fabdcecbc None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:17.375638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 745/1491] 10.222.0.67 () {54 vars in 926 bytes} [Fri Jun 24 15:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:13:17.380898 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de1a6f-7684-4162-9cd8-9da584293a6c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:17.382019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de1a6f-7684-4162-9cd8-9da584293a6c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:17.382019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de1a6f-7684-4162-9cd8-9da584293a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:17.399153 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f5de1a6f-7684-4162-9cd8-9da584293a6c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:13:17.400355 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f5de1a6f-7684-4162-9cd8-9da584293a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:13:17.407977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 747/1492] 10.222.0.67 () {58 vars in 1023 bytes} [Fri Jun 24 15:13:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:13:17.447346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b32c1-ae1d-4651-8740-1b3fabdcecbc None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:13:17.457831 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b32c1-ae1d-4651-8740-1b3fabdcecbc None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:13:17.461554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4139cf-cef8-4459-b622-c3a597ec5c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:13:17.462186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4139cf-cef8-4459-b622-c3a597ec5c45 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:13:17.462469 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4139cf-cef8-4459-b622-c3a597ec5c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:13:17.462744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4139cf-cef8-4459-b622-c3a597ec5c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:13:17.471583 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4139cf-cef8-4459-b622-c3a597ec5c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:13:17.472984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4139cf-cef8-4459-b622-c3a597ec5c45 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:13:17.482277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 746/1493] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:13:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:13:27.394135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=33 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:13:27.478155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=33 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:14:50.143533 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37b3ed-023b-4b3c-9502-940eb2901363 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:50.144105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37b3ed-023b-4b3c-9502-940eb2901363 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:50.144291 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37b3ed-023b-4b3c-9502-940eb2901363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:50.162685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ac37b3ed-023b-4b3c-9502-940eb2901363 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:14:50.163913 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ac37b3ed-023b-4b3c-9502-940eb2901363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:50.172194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 748/1494] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:14:50.180516 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6defcc0-9823-40d1-95cb-8e1a9e738810 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:50.181011 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6defcc0-9823-40d1-95cb-8e1a9e738810 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:50.181127 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6defcc0-9823-40d1-95cb-8e1a9e738810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:50.198710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e6defcc0-9823-40d1-95cb-8e1a9e738810 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:14:50.199950 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e6defcc0-9823-40d1-95cb-8e1a9e738810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:50.208040 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 747/1495] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:14:50.212506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37b3ed-023b-4b3c-9502-940eb2901363 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:50.222643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37b3ed-023b-4b3c-9502-940eb2901363 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:50.227050 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd8f8d-5024-4395-b4f0-7fcac54e76ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:50.227721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd8f8d-5024-4395-b4f0-7fcac54e76ff admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:50.228004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd8f8d-5024-4395-b4f0-7fcac54e76ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:50.228277 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd8f8d-5024-4395-b4f0-7fcac54e76ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:50.228986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afd8f8d-5024-4395-b4f0-7fcac54e76ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:14:50.229291 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afd8f8d-5024-4395-b4f0-7fcac54e76ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:50.230323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-1afd8f8d-5024-4395-b4f0-7fcac54e76ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 15:14:50.230323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jun 24 15:14:50.230323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: Jun 24 15:14:50.231206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afd8f8d-5024-4395-b4f0-7fcac54e76ff admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:50.242967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 749/1496] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:14:50.250707 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6defcc0-9823-40d1-95cb-8e1a9e738810 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:50.259902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6defcc0-9823-40d1-95cb-8e1a9e738810 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:50.264363 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14b5db-01d7-45ce-aa42-306ae46cb51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:50.265007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14b5db-01d7-45ce-aa42-306ae46cb51f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:50.265289 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14b5db-01d7-45ce-aa42-306ae46cb51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:50.265565 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14b5db-01d7-45ce-aa42-306ae46cb51f admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:50.266249 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14b5db-01d7-45ce-aa42-306ae46cb51f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:50.268034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-7e14b5db-01d7-45ce-aa42-306ae46cb51f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 15:14:50.268034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jun 24 15:14:50.268034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: Jun 24 15:14:50.269275 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14b5db-01d7-45ce-aa42-306ae46cb51f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:50.422851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 748/1497] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:14:50 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:14:50.428114 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd8f8d-5024-4395-b4f0-7fcac54e76ff admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:50.437268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd8f8d-5024-4395-b4f0-7fcac54e76ff admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:50.499852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8924a-0b5d-4dbb-b118-7b5df565623a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:50.500556 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8924a-0b5d-4dbb-b118-7b5df565623a admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:50.500796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8924a-0b5d-4dbb-b118-7b5df565623a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:50.501072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8924a-0b5d-4dbb-b118-7b5df565623a admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:50.501766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8924a-0b5d-4dbb-b118-7b5df565623a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:50.502775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-2cd8924a-0b5d-4dbb-b118-7b5df565623a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 15:14:50.502775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jun 24 15:14:50.502775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: Jun 24 15:14:50.504048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8924a-0b5d-4dbb-b118-7b5df565623a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:50.582379 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 750/1498] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:14:50 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:14:50.586499 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14b5db-01d7-45ce-aa42-306ae46cb51f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:50.596830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14b5db-01d7-45ce-aa42-306ae46cb51f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:50.602149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f19bd1-8aee-4627-bd94-9383e053df5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:50.602766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f19bd1-8aee-4627-bd94-9383e053df5c admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:50.603049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f19bd1-8aee-4627-bd94-9383e053df5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:50.603431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f19bd1-8aee-4627-bd94-9383e053df5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:50.604183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f19bd1-8aee-4627-bd94-9383e053df5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:50.605197 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-19f19bd1-8aee-4627-bd94-9383e053df5c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 15:14:50.605197 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jun 24 15:14:50.605197 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: Jun 24 15:14:50.606097 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f19bd1-8aee-4627-bd94-9383e053df5c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:50.619709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 749/1499] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:14:50.624130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8924a-0b5d-4dbb-b118-7b5df565623a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:50.634598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8924a-0b5d-4dbb-b118-7b5df565623a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:50.639086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9a5a5-87e1-4202-9c8d-cd1bc39d4264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:50.639786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9a5a5-87e1-4202-9c8d-cd1bc39d4264 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:50.640069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9a5a5-87e1-4202-9c8d-cd1bc39d4264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:50.640346 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9a5a5-87e1-4202-9c8d-cd1bc39d4264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:50.641070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9a5a5-87e1-4202-9c8d-cd1bc39d4264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:50.642176 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-8fd9a5a5-87e1-4202-9c8d-cd1bc39d4264 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 15:14:50.642176 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jun 24 15:14:50.642176 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: Jun 24 15:14:50.643069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9a5a5-87e1-4202-9c8d-cd1bc39d4264 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:50.657549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 751/1500] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:14:50.662688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f19bd1-8aee-4627-bd94-9383e053df5c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:50.672964 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f19bd1-8aee-4627-bd94-9383e053df5c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:50.678285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffca5f8-4f6c-4df0-baad-09a991f15b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:50.679250 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffca5f8-4f6c-4df0-baad-09a991f15b4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:50.679542 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffca5f8-4f6c-4df0-baad-09a991f15b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:50.679824 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffca5f8-4f6c-4df0-baad-09a991f15b4e admin admin] PATH_INFO: `/v3/projects/3bf2b74674ca4056910f80e346963761/users/ce9e3fd614cd4f1e95099edf9480d899/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:50.695811 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffca5f8-4f6c-4df0-baad-09a991f15b4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf2b74674ca4056910f80e346963761, user_id=ce9e3fd614cd4f1e95099edf9480d899, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:50.696853 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-cffca5f8-4f6c-4df0-baad-09a991f15b4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 15:14:50.696853 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jun 24 15:14:50.696853 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: Jun 24 15:14:50.698783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffca5f8-4f6c-4df0-baad-09a991f15b4e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:50.776314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 750/1501] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:14:50 2022] PUT /identity/v3/projects/3bf2b74674ca4056910f80e346963761/users/ce9e3fd614cd4f1e95099edf9480d899/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:14:50.787918 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d30cef-f19d-4055-93bd-ef3c8a9be414 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:50.788210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d30cef-f19d-4055-93bd-ef3c8a9be414 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:50.788519 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d30cef-f19d-4055-93bd-ef3c8a9be414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:50.814126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-27d30cef-f19d-4055-93bd-ef3c8a9be414 None None] MFA Rules not processed for user `ce9e3fd614cd4f1e95099edf9480d899`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:14:50.815399 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-27d30cef-f19d-4055-93bd-ef3c8a9be414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:50.878899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 752/1502] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:14:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:14:50.892212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42cfd20-7738-47df-8310-81a3bd90544f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:50.892505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42cfd20-7738-47df-8310-81a3bd90544f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:50.892789 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42cfd20-7738-47df-8310-81a3bd90544f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:50.895280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 751/1503] 10.222.0.67 () {54 vars in 963 bytes} [Fri Jun 24 15:14:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:14:50.899960 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484603cd-2d5d-4864-8056-5c84960c8708 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:50.900256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484603cd-2d5d-4864-8056-5c84960c8708 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:50.900539 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484603cd-2d5d-4864-8056-5c84960c8708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:50.918266 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-484603cd-2d5d-4864-8056-5c84960c8708 None None] MFA Rules not processed for user `2efb120c084a48858d32f97843d76834`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:14:50.919496 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-484603cd-2d5d-4864-8056-5c84960c8708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:50.961967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:14:50.969054 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 753/1504] 10.222.0.67 () {58 vars in 1060 bytes} [Fri Jun 24 15:14:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:14:50.977062 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42cfd20-7738-47df-8310-81a3bd90544f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:50.987591 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42cfd20-7738-47df-8310-81a3bd90544f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:50.991250 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04a877-740c-49ce-b99b-85b3422fa7c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:50.991917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04a877-740c-49ce-b99b-85b3422fa7c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:50.992204 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04a877-740c-49ce-b99b-85b3422fa7c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:50.992487 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04a877-740c-49ce-b99b-85b3422fa7c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:51.000717 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04a877-740c-49ce-b99b-85b3422fa7c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:51.002136 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04a877-740c-49ce-b99b-85b3422fa7c1 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:51.011578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 752/1505] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:14:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:14:51.053937 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ee2d7-5e52-4e60-91fb-23e85c28d3cf None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:51.054231 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ee2d7-5e52-4e60-91fb-23e85c28d3cf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:51.054519 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ee2d7-5e52-4e60-91fb-23e85c28d3cf None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:51.055842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 754/1506] 10.222.0.67 () {52 vars in 829 bytes} [Fri Jun 24 15:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:14:51.059460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04a877-740c-49ce-b99b-85b3422fa7c1 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:51.122928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04a877-740c-49ce-b99b-85b3422fa7c1 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:51.127471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df6080-8671-40c0-bad6-6c4731f6fe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:51.129073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df6080-8671-40c0-bad6-6c4731f6fe6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:51.129531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df6080-8671-40c0-bad6-6c4731f6fe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:51.129824 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df6080-8671-40c0-bad6-6c4731f6fe6a admin admin] PATH_INFO: `/v3/users/ce9e3fd614cd4f1e95099edf9480d899` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:51.131041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df6080-8671-40c0-bad6-6c4731f6fe6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9e3fd614cd4f1e95099edf9480d899)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:51.132088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-12df6080-8671-40c0-bad6-6c4731f6fe6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 15:14:51.132088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jun 24 15:14:51.132088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: Jun 24 15:14:51.134513 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df6080-8671-40c0-bad6-6c4731f6fe6a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:51.296772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-12df6080-8671-40c0-bad6-6c4731f6fe6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9e3fd614cd4f1e95099edf9480d899'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:51.317034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-12df6080-8671-40c0-bad6-6c4731f6fe6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9e3fd614cd4f1e95099edf9480d899'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:51.317931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-12df6080-8671-40c0-bad6-6c4731f6fe6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9e3fd614cd4f1e95099edf9480d899'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:51.330306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-12df6080-8671-40c0-bad6-6c4731f6fe6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9e3fd614cd4f1e95099edf9480d899'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:51.330972 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-12df6080-8671-40c0-bad6-6c4731f6fe6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9e3fd614cd4f1e95099edf9480d899'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:51.335299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:14:51.415608 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 753/1507] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:14:51 2022] DELETE /identity/v3/users/ce9e3fd614cd4f1e95099edf9480d899 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:14:51.420184 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eac41e-2e26-46f6-abfd-7c179ab00ac3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:51.420644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eac41e-2e26-46f6-abfd-7c179ab00ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:51.422872 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eac41e-2e26-46f6-abfd-7c179ab00ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:51.438730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a9eac41e-2e26-46f6-abfd-7c179ab00ac3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:14:51.440133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a9eac41e-2e26-46f6-abfd-7c179ab00ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:51.498472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 755/1508] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:14:51.508003 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df6080-8671-40c0-bad6-6c4731f6fe6a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:51.509405 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-12df6080-8671-40c0-bad6-6c4731f6fe6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:51.565622 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df6080-8671-40c0-bad6-6c4731f6fe6a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:51.569395 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54823ff-ab22-419b-bba3-1bcc529d7b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:51.570038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54823ff-ab22-419b-bba3-1bcc529d7b86 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:51.570323 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54823ff-ab22-419b-bba3-1bcc529d7b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:51.570602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54823ff-ab22-419b-bba3-1bcc529d7b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:51.579465 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54823ff-ab22-419b-bba3-1bcc529d7b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:51.580890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54823ff-ab22-419b-bba3-1bcc529d7b86 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:51.590187 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 754/1509] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:14:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:14:52.005601 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eac41e-2e26-46f6-abfd-7c179ab00ac3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:52.007027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a9eac41e-2e26-46f6-abfd-7c179ab00ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:52.019420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eac41e-2e26-46f6-abfd-7c179ab00ac3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:52.024383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:52.029934 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:52.030226 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:52.030508 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf admin admin] PATH_INFO: `/v3/projects/3bf2b74674ca4056910f80e346963761` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:52.031779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bf2b74674ca4056910f80e346963761)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:52.032815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 15:14:52.032815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jun 24 15:14:52.032815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: Jun 24 15:14:52.034030 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:52.139430 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bf2b74674ca4056910f80e346963761', 'request_id': 'req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:52.153195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bf2b74674ca4056910f80e346963761', 'request_id': 'req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:52.564272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bf2b74674ca4056910f80e346963761', 'request_id': 'req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:52.576363 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf admin admin] The token cache is being invalidate because project 3bf2b74674ca4056910f80e346963761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:14:52.576656 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:52.577401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d6e49bdb-f1aa-471a-8c6e-cccc5218e7cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:52.579022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 756/1510] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:14:52 2022] DELETE /identity/v3/projects/3bf2b74674ca4056910f80e346963761 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:15:01.586139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:15:02.573311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:16:32.007168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf40a75-0d63-4992-b787-895991daff0b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:32.007798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf40a75-0d63-4992-b787-895991daff0b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:32.007798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf40a75-0d63-4992-b787-895991daff0b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:32.009086 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 15:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:16:32.013825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e02a5e-defe-4a95-92c7-1e87f48ba54a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:32.014275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e02a5e-defe-4a95-92c7-1e87f48ba54a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:32.014380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e02a5e-defe-4a95-92c7-1e87f48ba54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:32.032792 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c4e02a5e-defe-4a95-92c7-1e87f48ba54a None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:16:32.033990 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c4e02a5e-defe-4a95-92c7-1e87f48ba54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:32.091928 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 15:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:16:32.246952 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf40a75-0d63-4992-b787-895991daff0b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:16:32.248900 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-edf40a75-0d63-4992-b787-895991daff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:32.325579 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf40a75-0d63-4992-b787-895991daff0b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:16:32.330786 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08be0b4-fb64-4f0d-a1d1-8eb77700695f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:16:32.335288 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08be0b4-fb64-4f0d-a1d1-8eb77700695f service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:32.335288 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08be0b4-fb64-4f0d-a1d1-8eb77700695f service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:32.335288 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08be0b4-fb64-4f0d-a1d1-8eb77700695f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:32.339914 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08be0b4-fb64-4f0d-a1d1-8eb77700695f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:16:32.341483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08be0b4-fb64-4f0d-a1d1-8eb77700695f service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:16:32.350934 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 756/1513] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:16:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:16:33.183785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf4c61-3597-4ae6-898e-0a77041bd72a None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:33.184341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf4c61-3597-4ae6-898e-0a77041bd72a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:33.184341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf4c61-3597-4ae6-898e-0a77041bd72a None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:33.185723 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 15:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:16:33.190508 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe84ab3-569d-42be-b4e8-73c63d7b3e47 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:33.190938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe84ab3-569d-42be-b4e8-73c63d7b3e47 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:33.191047 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe84ab3-569d-42be-b4e8-73c63d7b3e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:33.209168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-afe84ab3-569d-42be-b4e8-73c63d7b3e47 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:16:33.210345 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-afe84ab3-569d-42be-b4e8-73c63d7b3e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:33.217928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 15:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:16:33.368843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf4c61-3597-4ae6-898e-0a77041bd72a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:16:33.383220 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf4c61-3597-4ae6-898e-0a77041bd72a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:16:33.388457 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df17a6-38b4-421f-a14e-5062e21fa421 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:16:33.389093 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df17a6-38b4-421f-a14e-5062e21fa421 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:33.389377 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df17a6-38b4-421f-a14e-5062e21fa421 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:33.389651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df17a6-38b4-421f-a14e-5062e21fa421 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:33.398333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df17a6-38b4-421f-a14e-5062e21fa421 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:16:33.399970 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df17a6-38b4-421f-a14e-5062e21fa421 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:16:33.409170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 759/1516] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:16:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:16:34.028377 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fc32e-6b69-4bc8-8bd1-31dae4c23582 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:34.028667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fc32e-6b69-4bc8-8bd1-31dae4c23582 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:34.028950 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fc32e-6b69-4bc8-8bd1-31dae4c23582 None None] PATH_INFO: `/v3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:34.030270 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 758/1517] 10.222.0.64 () {54 vars in 925 bytes} [Fri Jun 24 15:16:34 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:16:34.036711 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30643c7-fae4-4acc-a81e-02d73eb9fb73 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:34.036999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30643c7-fae4-4acc-a81e-02d73eb9fb73 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:34.037275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30643c7-fae4-4acc-a81e-02d73eb9fb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:34.055214 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e30643c7-fae4-4acc-a81e-02d73eb9fb73 None None] MFA Rules not processed for user `f612d6ff06fc458e8c1178cbf8321f79`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:16:34.056413 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e30643c7-fae4-4acc-a81e-02d73eb9fb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:34.119492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 760/1518] 10.222.0.64 () {58 vars in 1015 bytes} [Fri Jun 24 15:16:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:16:38.085499 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db439226-b178-4ea2-8431-a1c1cb26e3fc None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:38.086016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db439226-b178-4ea2-8431-a1c1cb26e3fc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:38.086016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db439226-b178-4ea2-8431-a1c1cb26e3fc None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:38.087415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 759/1519] 10.222.0.67 () {54 vars in 960 bytes} [Fri Jun 24 15:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:16:38.092847 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6b4c6-a2fb-4a04-bbf2-57d711510d64 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:38.093311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6b4c6-a2fb-4a04-bbf2-57d711510d64 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:38.093421 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6b4c6-a2fb-4a04-bbf2-57d711510d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:38.112844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-46d6b4c6-a2fb-4a04-bbf2-57d711510d64 None None] MFA Rules not processed for user `f612d6ff06fc458e8c1178cbf8321f79`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:16:38.114042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-46d6b4c6-a2fb-4a04-bbf2-57d711510d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:38.122747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 761/1520] 10.222.0.67 () {58 vars in 1057 bytes} [Fri Jun 24 15:16:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:16:38.130891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db439226-b178-4ea2-8431-a1c1cb26e3fc None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:16:38.144260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db439226-b178-4ea2-8431-a1c1cb26e3fc None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:16:38.150242 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f52aa5-3bc5-409f-bad6-b771badd1b8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:16:38.150903 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f52aa5-3bc5-409f-bad6-b771badd1b8c service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:38.151207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f52aa5-3bc5-409f-bad6-b771badd1b8c service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:38.151564 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f52aa5-3bc5-409f-bad6-b771badd1b8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:38.160056 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f52aa5-3bc5-409f-bad6-b771badd1b8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:16:38.161759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f52aa5-3bc5-409f-bad6-b771badd1b8c service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:16:38.171532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 760/1521] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:16:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:16:44.670257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf063089-b86a-452a-a6ea-94e7f3af94dd None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:44.670774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf063089-b86a-452a-a6ea-94e7f3af94dd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:44.670774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf063089-b86a-452a-a6ea-94e7f3af94dd None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:44.672166 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 15:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:16:44.677672 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfcfa8-d5db-4c76-a8d2-883ebff8f600 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:44.678320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfcfa8-d5db-4c76-a8d2-883ebff8f600 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:44.678430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfcfa8-d5db-4c76-a8d2-883ebff8f600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:44.696050 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a4dfcfa8-d5db-4c76-a8d2-883ebff8f600 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:16:44.697247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a4dfcfa8-d5db-4c76-a8d2-883ebff8f600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:44.705021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 15:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:16:44.861899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf063089-b86a-452a-a6ea-94e7f3af94dd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:16:44.876918 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf063089-b86a-452a-a6ea-94e7f3af94dd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:16:44.882735 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88f346-700f-4880-ac84-65884b6667d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:16:44.883408 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88f346-700f-4880-ac84-65884b6667d2 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:44.883696 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88f346-700f-4880-ac84-65884b6667d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:44.883987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88f346-700f-4880-ac84-65884b6667d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:44.892390 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88f346-700f-4880-ac84-65884b6667d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:16:44.894014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88f346-700f-4880-ac84-65884b6667d2 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:16:44.903796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 763/1524] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:16:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:16:45.723507 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f68f7-9f46-49ab-a9db-a369e06acd01 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:45.724122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f68f7-9f46-49ab-a9db-a369e06acd01 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:45.724122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f68f7-9f46-49ab-a9db-a369e06acd01 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:45.725469 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 15:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:16:45.730213 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9efaf2-804d-4b88-a102-9a5ccf076314 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:45.730669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9efaf2-804d-4b88-a102-9a5ccf076314 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:45.730781 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9efaf2-804d-4b88-a102-9a5ccf076314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:45.749560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8b9efaf2-804d-4b88-a102-9a5ccf076314 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:16:45.750753 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8b9efaf2-804d-4b88-a102-9a5ccf076314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:45.759701 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 15:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:16:45.916163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f68f7-9f46-49ab-a9db-a369e06acd01 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:16:45.931009 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f68f7-9f46-49ab-a9db-a369e06acd01 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:16:45.936454 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe917470-a27f-44b0-9ad3-d74eb0f50c18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:16:45.937091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe917470-a27f-44b0-9ad3-d74eb0f50c18 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:45.937373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe917470-a27f-44b0-9ad3-d74eb0f50c18 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:45.937650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe917470-a27f-44b0-9ad3-d74eb0f50c18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:45.946223 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe917470-a27f-44b0-9ad3-d74eb0f50c18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:16:45.947829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe917470-a27f-44b0-9ad3-d74eb0f50c18 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:16:45.957088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 763/1527] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:16:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:16:50.004180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f910359-c873-4746-a560-6a6e1c03eb21 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:50.004721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f910359-c873-4746-a560-6a6e1c03eb21 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:50.004721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f910359-c873-4746-a560-6a6e1c03eb21 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:50.006601 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 15:16:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:16:50.012474 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a41f04-cb72-45d7-809c-83346199a596 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:50.012917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a41f04-cb72-45d7-809c-83346199a596 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:50.013023 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a41f04-cb72-45d7-809c-83346199a596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:50.033577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-43a41f04-cb72-45d7-809c-83346199a596 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:16:50.035075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-43a41f04-cb72-45d7-809c-83346199a596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:50.043755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 15:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:16:50.090341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f910359-c873-4746-a560-6a6e1c03eb21 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:16:50.092323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2f910359-c873-4746-a560-6a6e1c03eb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:50.148814 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f910359-c873-4746-a560-6a6e1c03eb21 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:16:50.152440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88ac28-574e-4bfa-b138-4efeaa86e402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:16:50.153094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88ac28-574e-4bfa-b138-4efeaa86e402 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:50.153378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88ac28-574e-4bfa-b138-4efeaa86e402 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:50.153661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88ac28-574e-4bfa-b138-4efeaa86e402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:50.163063 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88ac28-574e-4bfa-b138-4efeaa86e402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:16:50.164747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88ac28-574e-4bfa-b138-4efeaa86e402 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:16:50.169655 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:16:50.175307 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 766/1530] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:16:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:16:52.253292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e61d79-d71f-4e1a-ab3b-2e22dccd7cc8 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:52.253849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e61d79-d71f-4e1a-ab3b-2e22dccd7cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:52.253849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e61d79-d71f-4e1a-ab3b-2e22dccd7cc8 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:52.255337 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 15:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:16:52.260900 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de503a-4aa7-49e0-936e-4b8df15722c0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:52.261418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de503a-4aa7-49e0-936e-4b8df15722c0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:52.261538 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de503a-4aa7-49e0-936e-4b8df15722c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:52.279526 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e4de503a-4aa7-49e0-936e-4b8df15722c0 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:16:52.280791 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e4de503a-4aa7-49e0-936e-4b8df15722c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:52.289460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 15:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:16:52.332714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e61d79-d71f-4e1a-ab3b-2e22dccd7cc8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:16:52.344807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e61d79-d71f-4e1a-ab3b-2e22dccd7cc8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:16:52.348305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1686183-90a5-439f-9bed-ea2678cbade8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:16:52.348946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1686183-90a5-439f-9bed-ea2678cbade8 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:52.349224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1686183-90a5-439f-9bed-ea2678cbade8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:52.349495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1686183-90a5-439f-9bed-ea2678cbade8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:52.359402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1686183-90a5-439f-9bed-ea2678cbade8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:16:52.360829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1686183-90a5-439f-9bed-ea2678cbade8 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:16:52.371351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 766/1533] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:16:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:16:56.063933 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be89ae-a842-4b5c-a6d2-2421effccbd6 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:56.064606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be89ae-a842-4b5c-a6d2-2421effccbd6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:56.064606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be89ae-a842-4b5c-a6d2-2421effccbd6 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:56.065858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 15:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:16:56.071710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3266fbd-ffdc-44c1-9ae3-95d6f1ecd4b9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:56.072133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3266fbd-ffdc-44c1-9ae3-95d6f1ecd4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:56.072275 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3266fbd-ffdc-44c1-9ae3-95d6f1ecd4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:56.091026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b3266fbd-ffdc-44c1-9ae3-95d6f1ecd4b9 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:16:56.092282 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b3266fbd-ffdc-44c1-9ae3-95d6f1ecd4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:56.100139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 15:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:16:56.404878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be89ae-a842-4b5c-a6d2-2421effccbd6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:16:56.406560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c4be89ae-a842-4b5c-a6d2-2421effccbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:56.463859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be89ae-a842-4b5c-a6d2-2421effccbd6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:16:56.467537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c52c9-1f85-4644-be69-6dab1239f433 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:16:56.468193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c52c9-1f85-4644-be69-6dab1239f433 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:56.468481 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c52c9-1f85-4644-be69-6dab1239f433 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:56.468762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c52c9-1f85-4644-be69-6dab1239f433 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:56.477883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c52c9-1f85-4644-be69-6dab1239f433 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:16:56.479356 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c52c9-1f85-4644-be69-6dab1239f433 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:16:56.488744 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 769/1536] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:16:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:16:58.564512 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3e1f4-609e-48d9-a53a-058a91022540 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:58.565061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3e1f4-609e-48d9-a53a-058a91022540 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:58.565061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3e1f4-609e-48d9-a53a-058a91022540 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:58.566442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 15:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:16:58.571710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524424ec-fe33-4907-9002-1fdad2f31684 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:58.572162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524424ec-fe33-4907-9002-1fdad2f31684 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:58.572290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524424ec-fe33-4907-9002-1fdad2f31684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:58.590452 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-524424ec-fe33-4907-9002-1fdad2f31684 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:16:58.591680 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-524424ec-fe33-4907-9002-1fdad2f31684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:58.600099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 15:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:16:58.907292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3e1f4-609e-48d9-a53a-058a91022540 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:16:58.911285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-eac3e1f4-609e-48d9-a53a-058a91022540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:58.922960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3e1f4-609e-48d9-a53a-058a91022540 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:16:58.926575 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c2267-94c9-4103-9b4d-168b4dee7392 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:16:58.927238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c2267-94c9-4103-9b4d-168b4dee7392 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:58.927555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c2267-94c9-4103-9b4d-168b4dee7392 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:58.927827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c2267-94c9-4103-9b4d-168b4dee7392 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:58.936678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c2267-94c9-4103-9b4d-168b4dee7392 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:16:58.938093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c2267-94c9-4103-9b4d-168b4dee7392 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:16:58.948222 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 769/1539] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:16:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:17:08.585912 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=32 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:17:08.943302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=70 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:27.060329 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d544c-e670-48e7-b751-e7904a77bab9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.079331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d544c-e670-48e7-b751-e7904a77bab9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.079331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d544c-e670-48e7-b751-e7904a77bab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.083893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-987d544c-e670-48e7-b751-e7904a77bab9 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:27.085375 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-987d544c-e670-48e7-b751-e7904a77bab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:27.098828 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 771/1540] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:27.108210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480f935-a3e9-4df5-b8d4-10317cf48e8f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.109182 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480f935-a3e9-4df5-b8d4-10317cf48e8f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.109680 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480f935-a3e9-4df5-b8d4-10317cf48e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.131925 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d480f935-a3e9-4df5-b8d4-10317cf48e8f None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:27.133473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d480f935-a3e9-4df5-b8d4-10317cf48e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:27.147594 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 770/1541] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:27.152637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d544c-e670-48e7-b751-e7904a77bab9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:27.164445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d544c-e670-48e7-b751-e7904a77bab9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:27.172856 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8f80d-9334-449e-9e13-e839b7dde54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:27.173830 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8f80d-9334-449e-9e13-e839b7dde54a admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.174343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8f80d-9334-449e-9e13-e839b7dde54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.174839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8f80d-9334-449e-9e13-e839b7dde54a admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.175887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8f80d-9334-449e-9e13-e839b7dde54a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:18:27.176423 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8f80d-9334-449e-9e13-e839b7dde54a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:27.178386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8f80d-9334-449e-9e13-e839b7dde54a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:27.194085 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 772/1542] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:18:27.202546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480f935-a3e9-4df5-b8d4-10317cf48e8f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:27.214081 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480f935-a3e9-4df5-b8d4-10317cf48e8f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:27.219164 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e4f0e-58c9-4577-b74b-ebcc3ede93c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:27.220239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e4f0e-58c9-4577-b74b-ebcc3ede93c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.220746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e4f0e-58c9-4577-b74b-ebcc3ede93c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.221241 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e4f0e-58c9-4577-b74b-ebcc3ede93c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.222169 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e4f0e-58c9-4577-b74b-ebcc3ede93c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:27.224973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e4f0e-58c9-4577-b74b-ebcc3ede93c3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:27.328761 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 771/1543] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:27.334014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8f80d-9334-449e-9e13-e839b7dde54a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:27.340847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b6897-ce84-4df6-82d6-139e808e6f31 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.340847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b6897-ce84-4df6-82d6-139e808e6f31 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.340847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b6897-ce84-4df6-82d6-139e808e6f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.349702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8f80d-9334-449e-9e13-e839b7dde54a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:27.366814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-786b6897-ce84-4df6-82d6-139e808e6f31 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:27.368285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-786b6897-ce84-4df6-82d6-139e808e6f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:27.468883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfab25-c2e4-4520-a829-ceb825075323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:27.470191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfab25-c2e4-4520-a829-ceb825075323 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.470931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfab25-c2e4-4520-a829-ceb825075323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.471453 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfab25-c2e4-4520-a829-ceb825075323 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.472404 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfab25-c2e4-4520-a829-ceb825075323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:27.475445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfab25-c2e4-4520-a829-ceb825075323 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:27.481358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 772/1544] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:18:27.496671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a417d2a-197e-43a7-836d-af95f43cc499 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.497021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a417d2a-197e-43a7-836d-af95f43cc499 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.497292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a417d2a-197e-43a7-836d-af95f43cc499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.535144 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4a417d2a-197e-43a7-836d-af95f43cc499 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:27.537029 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4a417d2a-197e-43a7-836d-af95f43cc499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:27.560283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 773/1545] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:27.561243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 773/1546] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/users => generated 290 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:27.565202 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a417d2a-197e-43a7-836d-af95f43cc499 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:27.565982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98434558-e76a-4d0a-879c-1e7173a3c907 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.566494 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98434558-e76a-4d0a-879c-1e7173a3c907 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.566991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98434558-e76a-4d0a-879c-1e7173a3c907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.579005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a417d2a-197e-43a7-836d-af95f43cc499 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:27.593627 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-98434558-e76a-4d0a-879c-1e7173a3c907 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:27.594339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb24cb8-1926-49f1-a2d3-c854d56510d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:27.595170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb24cb8-1926-49f1-a2d3-c854d56510d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.595485 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb24cb8-1926-49f1-a2d3-c854d56510d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.595771 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb24cb8-1926-49f1-a2d3-c854d56510d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.596378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-98434558-e76a-4d0a-879c-1e7173a3c907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:27.596492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb24cb8-1926-49f1-a2d3-c854d56510d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:18:27.596812 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb24cb8-1926-49f1-a2d3-c854d56510d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:27.602474 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-ebb24cb8-1926-49f1-a2d3-c854d56510d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 15:18:27.602474 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jun 24 15:18:27.602474 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: Jun 24 15:18:27.602474 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb24cb8-1926-49f1-a2d3-c854d56510d3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:27.616715 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 774/1547] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:18:27.617913 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98434558-e76a-4d0a-879c-1e7173a3c907 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:27.621249 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 774/1548] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:18:27.622563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc50e0-e632-42a8-8f95-edd5e089ac05 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.622859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc50e0-e632-42a8-8f95-edd5e089ac05 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.623139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc50e0-e632-42a8-8f95-edd5e089ac05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.634123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98434558-e76a-4d0a-879c-1e7173a3c907 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:27.642174 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5bec6-50e2-4861-988c-a9e735512926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:27.643212 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5bec6-50e2-4861-988c-a9e735512926 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.643773 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5bec6-50e2-4861-988c-a9e735512926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.644273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5bec6-50e2-4861-988c-a9e735512926 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.645247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5bec6-50e2-4861-988c-a9e735512926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:27.648016 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5bec6-50e2-4861-988c-a9e735512926 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:27.659526 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 775/1549] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:27.660699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5bec6-50e2-4861-988c-a9e735512926 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:27.672157 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-30cc50e0-e632-42a8-8f95-edd5e089ac05 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:27.677234 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-30cc50e0-e632-42a8-8f95-edd5e089ac05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:27.682389 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5bec6-50e2-4861-988c-a9e735512926 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:27.693751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c065cb-0cb2-4832-b697-6159d34a67ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:27.694796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c065cb-0cb2-4832-b697-6159d34a67ae admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.695615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 775/1550] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:27.697135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd016154-9daa-443d-9151-0d44fc3be4fb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.697419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c065cb-0cb2-4832-b697-6159d34a67ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.697935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c065cb-0cb2-4832-b697-6159d34a67ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.699000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c065cb-0cb2-4832-b697-6159d34a67ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:27.700290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-14c065cb-0cb2-4832-b697-6159d34a67ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 15:18:27.700290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jun 24 15:18:27.700290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: Jun 24 15:18:27.700978 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd016154-9daa-443d-9151-0d44fc3be4fb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.701722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd016154-9daa-443d-9151-0d44fc3be4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.703583 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c065cb-0cb2-4832-b697-6159d34a67ae admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:27.732643 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-cd016154-9daa-443d-9151-0d44fc3be4fb None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:27.733928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-cd016154-9daa-443d-9151-0d44fc3be4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:27.743990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 776/1551] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:27.745303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00df89-ecb1-4a3b-b522-43076a2407eb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.745609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00df89-ecb1-4a3b-b522-43076a2407eb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.745885 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00df89-ecb1-4a3b-b522-43076a2407eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.764730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ac00df89-ecb1-4a3b-b522-43076a2407eb None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:27.766752 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ac00df89-ecb1-4a3b-b522-43076a2407eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:27.774959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 777/1552] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:27.775783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00df89-ecb1-4a3b-b522-43076a2407eb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:27.785348 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00df89-ecb1-4a3b-b522-43076a2407eb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:27.790516 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41690862-4ee0-404b-94d6-d477fe96a87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:27.791198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41690862-4ee0-404b-94d6-d477fe96a87e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.791485 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41690862-4ee0-404b-94d6-d477fe96a87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.791762 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41690862-4ee0-404b-94d6-d477fe96a87e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.792490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41690862-4ee0-404b-94d6-d477fe96a87e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:27.794098 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41690862-4ee0-404b-94d6-d477fe96a87e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:27.801064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 776/1553] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:27.802709 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbc894-5c1f-486b-bf69-68e5390b44d8 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.802999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbc894-5c1f-486b-bf69-68e5390b44d8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.803308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbc894-5c1f-486b-bf69-68e5390b44d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.808036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 778/1554] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:27.808036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82ee48-230e-468d-a1ff-af86a882f90e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.808036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82ee48-230e-468d-a1ff-af86a882f90e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.808036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82ee48-230e-468d-a1ff-af86a882f90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.821718 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-33dbc894-5c1f-486b-bf69-68e5390b44d8 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:27.822939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-33dbc894-5c1f-486b-bf69-68e5390b44d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:27.825077 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0f82ee48-230e-468d-a1ff-af86a882f90e None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:27.826250 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0f82ee48-230e-468d-a1ff-af86a882f90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:27.885995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 777/1555] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:27.885995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4af8b-82a2-46e5-9f41-3f825bdac622 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.885995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4af8b-82a2-46e5-9f41-3f825bdac622 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.886334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 779/1556] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:27.886447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4af8b-82a2-46e5-9f41-3f825bdac622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.886566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82ee48-230e-468d-a1ff-af86a882f90e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:27.901170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82ee48-230e-468d-a1ff-af86a882f90e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:27.905884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b423e-392d-4923-969c-56034b088544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:27.906600 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b423e-392d-4923-969c-56034b088544 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.907135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b423e-392d-4923-969c-56034b088544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.907451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b423e-392d-4923-969c-56034b088544 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.908194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b423e-392d-4923-969c-56034b088544 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:18:27.908506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b423e-392d-4923-969c-56034b088544 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:27.910133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b423e-392d-4923-969c-56034b088544 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:27.923701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 780/1557] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:18:27.925058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec7851-25a9-4a84-93c6-e066d5e1fa49 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.925368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec7851-25a9-4a84-93c6-e066d5e1fa49 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.925636 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec7851-25a9-4a84-93c6-e066d5e1fa49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.944842 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9fec7851-25a9-4a84-93c6-e066d5e1fa49 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:27.946101 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9fec7851-25a9-4a84-93c6-e066d5e1fa49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:27.954321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 781/1558] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:27.955700 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcdf5c-be6f-4729-9787-b22ba211c937 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:27.955992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcdf5c-be6f-4729-9787-b22ba211c937 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:27.956272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcdf5c-be6f-4729-9787-b22ba211c937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:27.963914 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:27.977119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-fbdcdf5c-be6f-4729-9787-b22ba211c937 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:27.978357 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fbdcdf5c-be6f-4729-9787-b22ba211c937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:27.987340 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 782/1559] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:27.988590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdcdf5c-be6f-4729-9787-b22ba211c937 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:27.999734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdcdf5c-be6f-4729-9787-b22ba211c937 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.004389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048adaed-24ea-4059-a548-dff9ca2cd7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.005832 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048adaed-24ea-4059-a548-dff9ca2cd7b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.006175 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048adaed-24ea-4059-a548-dff9ca2cd7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.006481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048adaed-24ea-4059-a548-dff9ca2cd7b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.007152 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048adaed-24ea-4059-a548-dff9ca2cd7b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.008312 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-048adaed-24ea-4059-a548-dff9ca2cd7b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 15:18:28.008312 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jun 24 15:18:28.008312 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: Jun 24 15:18:28.009515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048adaed-24ea-4059-a548-dff9ca2cd7b0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.023663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9cd4af8b-82a2-46e5-9f41-3f825bdac622 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:28.024951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9cd4af8b-82a2-46e5-9f41-3f825bdac622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:28.033130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 778/1560] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:28.033862 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4af8b-82a2-46e5-9f41-3f825bdac622 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.043238 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4af8b-82a2-46e5-9f41-3f825bdac622 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.047904 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc27a3-af33-4fb6-b303-7c38d7ab5339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.053270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc27a3-af33-4fb6-b303-7c38d7ab5339 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.053270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc27a3-af33-4fb6-b303-7c38d7ab5339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.053270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc27a3-af33-4fb6-b303-7c38d7ab5339 admin admin] PATH_INFO: `/v3/projects/d6e6bc2109c44c37a0ed635b021bba7e/users/78195ff60f0242ada974a8e08d1f2754/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.063705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc27a3-af33-4fb6-b303-7c38d7ab5339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e6bc2109c44c37a0ed635b021bba7e, user_id=78195ff60f0242ada974a8e08d1f2754, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.064772 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-addc27a3-af33-4fb6-b303-7c38d7ab5339 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 15:18:28.064772 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jun 24 15:18:28.064772 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: Jun 24 15:18:28.066709 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc27a3-af33-4fb6-b303-7c38d7ab5339 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.080056 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 783/1561] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:27 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:28.080871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048adaed-24ea-4059-a548-dff9ca2cd7b0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.091105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048adaed-24ea-4059-a548-dff9ca2cd7b0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.095792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a5b48-f53b-4a9a-ba82-1efa1dcee642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.096438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a5b48-f53b-4a9a-ba82-1efa1dcee642 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.096722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a5b48-f53b-4a9a-ba82-1efa1dcee642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.097014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a5b48-f53b-4a9a-ba82-1efa1dcee642 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.097729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a5b48-f53b-4a9a-ba82-1efa1dcee642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:18:28.098063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a5b48-f53b-4a9a-ba82-1efa1dcee642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.099738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a5b48-f53b-4a9a-ba82-1efa1dcee642 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.111601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 784/1562] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:18:28.112836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456dd93-6343-40ce-8b93-bc91613e80e6 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.113126 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456dd93-6343-40ce-8b93-bc91613e80e6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.113400 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456dd93-6343-40ce-8b93-bc91613e80e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.126994 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:28.131542 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0456dd93-6343-40ce-8b93-bc91613e80e6 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:28.133011 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0456dd93-6343-40ce-8b93-bc91613e80e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:28.133145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 779/1563] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:28 2022] PUT /identity/v3/projects/d6e6bc2109c44c37a0ed635b021bba7e/users/78195ff60f0242ada974a8e08d1f2754/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:28.133792 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc27a3-af33-4fb6-b303-7c38d7ab5339 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.195741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 785/1564] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:28.197018 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584581f-677b-447c-935d-f6576f68bccb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.197314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584581f-677b-447c-935d-f6576f68bccb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.197593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584581f-677b-447c-935d-f6576f68bccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.198185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc27a3-af33-4fb6-b303-7c38d7ab5339 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.203024 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e5ebe-b921-497e-bd09-76b8a5af4660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.203912 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e5ebe-b921-497e-bd09-76b8a5af4660 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.206374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e5ebe-b921-497e-bd09-76b8a5af4660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.206374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e5ebe-b921-497e-bd09-76b8a5af4660 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.206374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e5ebe-b921-497e-bd09-76b8a5af4660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.207062 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e5ebe-b921-497e-bd09-76b8a5af4660 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.216024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6584581f-677b-447c-935d-f6576f68bccb None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:28.217224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6584581f-677b-447c-935d-f6576f68bccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:28.225560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 786/1565] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:28.226542 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584581f-677b-447c-935d-f6576f68bccb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.236083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584581f-677b-447c-935d-f6576f68bccb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.240984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bf18c-88a8-4410-b007-37a74bfc3295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.241629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bf18c-88a8-4410-b007-37a74bfc3295 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.241912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bf18c-88a8-4410-b007-37a74bfc3295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.242188 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bf18c-88a8-4410-b007-37a74bfc3295 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.242946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bf18c-88a8-4410-b007-37a74bfc3295 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:18:28.243292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bf18c-88a8-4410-b007-37a74bfc3295 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.244925 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bf18c-88a8-4410-b007-37a74bfc3295 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.256938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 787/1566] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:18:28.258122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1f6f2-2cb0-4b23-8a8d-8be61a82ac43 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.258409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1f6f2-2cb0-4b23-8a8d-8be61a82ac43 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.258694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1f6f2-2cb0-4b23-8a8d-8be61a82ac43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.276653 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b7c1f6f2-2cb0-4b23-8a8d-8be61a82ac43 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:28.277845 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b7c1f6f2-2cb0-4b23-8a8d-8be61a82ac43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:28.285969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 788/1567] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:28.286841 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1f6f2-2cb0-4b23-8a8d-8be61a82ac43 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.295912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1f6f2-2cb0-4b23-8a8d-8be61a82ac43 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.298925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 780/1568] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:28 2022] POST /identity/v3/projects => generated 367 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:28.299773 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e5ebe-b921-497e-bd09-76b8a5af4660 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.309449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44fb12-6e11-49a9-afe7-f393e8df07a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.309449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44fb12-6e11-49a9-afe7-f393e8df07a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.309449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44fb12-6e11-49a9-afe7-f393e8df07a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.309449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44fb12-6e11-49a9-afe7-f393e8df07a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.309832 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44fb12-6e11-49a9-afe7-f393e8df07a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.309832 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44fb12-6e11-49a9-afe7-f393e8df07a4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.314665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 789/1569] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:28.316113 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30bda4-3644-4853-a9af-54164947ec0a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.316404 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30bda4-3644-4853-a9af-54164947ec0a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.316683 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30bda4-3644-4853-a9af-54164947ec0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.334511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0a30bda4-3644-4853-a9af-54164947ec0a None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:28.335768 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0a30bda4-3644-4853-a9af-54164947ec0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:28.361632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e5ebe-b921-497e-bd09-76b8a5af4660 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.366713 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2d604-96e0-42b7-a444-c3659492ff96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.367320 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2d604-96e0-42b7-a444-c3659492ff96 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.367602 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2d604-96e0-42b7-a444-c3659492ff96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.367882 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2d604-96e0-42b7-a444-c3659492ff96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.368555 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2d604-96e0-42b7-a444-c3659492ff96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.370431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b2d604-96e0-42b7-a444-c3659492ff96 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.395334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 790/1570] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:28.396351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30bda4-3644-4853-a9af-54164947ec0a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.405735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30bda4-3644-4853-a9af-54164947ec0a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.410682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e860831-e941-4153-b5db-2b19b86d353c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.411751 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e860831-e941-4153-b5db-2b19b86d353c admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.412078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e860831-e941-4153-b5db-2b19b86d353c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.412362 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e860831-e941-4153-b5db-2b19b86d353c admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.413082 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e860831-e941-4153-b5db-2b19b86d353c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:18:28.413396 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e860831-e941-4153-b5db-2b19b86d353c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.414988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e860831-e941-4153-b5db-2b19b86d353c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.426319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 791/1571] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:18:28.427823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e860831-e941-4153-b5db-2b19b86d353c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.436994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e860831-e941-4153-b5db-2b19b86d353c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.441596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5914513-d523-48c7-a312-20a07d3a6bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.442210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5914513-d523-48c7-a312-20a07d3a6bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.443206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5914513-d523-48c7-a312-20a07d3a6bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.443495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5914513-d523-48c7-a312-20a07d3a6bb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.444281 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5914513-d523-48c7-a312-20a07d3a6bb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:18:28.444588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5914513-d523-48c7-a312-20a07d3a6bb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.446188 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5914513-d523-48c7-a312-20a07d3a6bb1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.457977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 792/1572] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:18:28.458808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5914513-d523-48c7-a312-20a07d3a6bb1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.463535 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:28.469779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 781/1573] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:28 2022] POST /identity/v3/projects => generated 367 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:28.470092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5914513-d523-48c7-a312-20a07d3a6bb1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.470642 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2d604-96e0-42b7-a444-c3659492ff96 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.527429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7d287-1026-4a7d-9fb9-c4c5ccfb93b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.528091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7d287-1026-4a7d-9fb9-c4c5ccfb93b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.528372 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7d287-1026-4a7d-9fb9-c4c5ccfb93b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.528646 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7d287-1026-4a7d-9fb9-c4c5ccfb93b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.529341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7d287-1026-4a7d-9fb9-c4c5ccfb93b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.531277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7d287-1026-4a7d-9fb9-c4c5ccfb93b8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.533233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2d604-96e0-42b7-a444-c3659492ff96 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.538359 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b4494-737f-44de-9873-fab971077fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.539045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b4494-737f-44de-9873-fab971077fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.539392 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b4494-737f-44de-9873-fab971077fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.539671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b4494-737f-44de-9873-fab971077fa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.540491 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b4494-737f-44de-9873-fab971077fa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:18:28.540797 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b4494-737f-44de-9873-fab971077fa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.542422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b4494-737f-44de-9873-fab971077fa6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.554500 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 782/1574] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:18:28.555807 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b4494-737f-44de-9873-fab971077fa6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.561202 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:28.565101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b4494-737f-44de-9873-fab971077fa6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.570213 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4aa295-d4d4-4cde-a94c-bc9a96188f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.570861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4aa295-d4d4-4cde-a94c-bc9a96188f1e admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.571148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4aa295-d4d4-4cde-a94c-bc9a96188f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.571465 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4aa295-d4d4-4cde-a94c-bc9a96188f1e admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.572145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4aa295-d4d4-4cde-a94c-bc9a96188f1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.574044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4aa295-d4d4-4cde-a94c-bc9a96188f1e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.635506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 793/1575] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:28 2022] POST /identity/v3/projects => generated 393 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:28.636388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7d287-1026-4a7d-9fb9-c4c5ccfb93b8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.664303 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 783/1576] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:28 2022] POST /identity/v3/users => generated 284 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:28.665224 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4aa295-d4d4-4cde-a94c-bc9a96188f1e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.700274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7d287-1026-4a7d-9fb9-c4c5ccfb93b8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.705043 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb6ce-7188-4553-aa2d-49351cae1e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.705717 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb6ce-7188-4553-aa2d-49351cae1e68 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.706005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb6ce-7188-4553-aa2d-49351cae1e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.706288 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb6ce-7188-4553-aa2d-49351cae1e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.707016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cb6ce-7188-4553-aa2d-49351cae1e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.709201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cb6ce-7188-4553-aa2d-49351cae1e68 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.718704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4aa295-d4d4-4cde-a94c-bc9a96188f1e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.719506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 794/1577] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:28.720195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb6ce-7188-4553-aa2d-49351cae1e68 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.730531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb6ce-7188-4553-aa2d-49351cae1e68 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.735204 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7eb47b-e85e-46b8-8c55-035011a26edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.735832 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7eb47b-e85e-46b8-8c55-035011a26edf admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.736118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7eb47b-e85e-46b8-8c55-035011a26edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.736400 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7eb47b-e85e-46b8-8c55-035011a26edf admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.737087 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7eb47b-e85e-46b8-8c55-035011a26edf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.739032 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7eb47b-e85e-46b8-8c55-035011a26edf admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.769553 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c185c62-9463-4455-995f-1dd352b1185a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.770200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c185c62-9463-4455-995f-1dd352b1185a service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.771121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c185c62-9463-4455-995f-1dd352b1185a service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.771434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c185c62-9463-4455-995f-1dd352b1185a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.780313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c185c62-9463-4455-995f-1dd352b1185a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.781722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c185c62-9463-4455-995f-1dd352b1185a service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.791256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 784/1578] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:28.792083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c185c62-9463-4455-995f-1dd352b1185a service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.800939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c185c62-9463-4455-995f-1dd352b1185a service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.805566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9741963-37b7-4293-86bf-2d7a093c8f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.806208 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9741963-37b7-4293-86bf-2d7a093c8f11 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.806488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9741963-37b7-4293-86bf-2d7a093c8f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.808700 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9741963-37b7-4293-86bf-2d7a093c8f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.808700 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9741963-37b7-4293-86bf-2d7a093c8f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.809317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9741963-37b7-4293-86bf-2d7a093c8f11 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:28.858303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 795/1579] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:28 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:28.859303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7eb47b-e85e-46b8-8c55-035011a26edf admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.890873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 785/1580] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:28 2022] POST /identity/v3/projects => generated 363 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:28.890873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9741963-37b7-4293-86bf-2d7a093c8f11 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:28.938707 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7eb47b-e85e-46b8-8c55-035011a26edf admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.944742 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:28.959621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9741963-37b7-4293-86bf-2d7a093c8f11 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:28.964296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf4c0b-86ff-4a0c-ad0a-409eebc07175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:28.969567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf4c0b-86ff-4a0c-ad0a-409eebc07175 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:28.969567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf4c0b-86ff-4a0c-ad0a-409eebc07175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:28.969567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf4c0b-86ff-4a0c-ad0a-409eebc07175 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:28.969567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf4c0b-86ff-4a0c-ad0a-409eebc07175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:28.969567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf4c0b-86ff-4a0c-ad0a-409eebc07175 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.001418 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb018c-144f-4c85-8198-a7015c2217e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.002098 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb018c-144f-4c85-8198-a7015c2217e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.002384 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb018c-144f-4c85-8198-a7015c2217e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.002669 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb018c-144f-4c85-8198-a7015c2217e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.003392 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb018c-144f-4c85-8198-a7015c2217e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.005458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb018c-144f-4c85-8198-a7015c2217e3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.056561 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 786/1581] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:28 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:29.056561 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf4c0b-86ff-4a0c-ad0a-409eebc07175 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.066158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:29.124257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf4c0b-86ff-4a0c-ad0a-409eebc07175 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.129202 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971530ae-ebe8-4dff-aee2-b6a7914b2bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.135445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971530ae-ebe8-4dff-aee2-b6a7914b2bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.135445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971530ae-ebe8-4dff-aee2-b6a7914b2bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.135445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971530ae-ebe8-4dff-aee2-b6a7914b2bc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.135445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971530ae-ebe8-4dff-aee2-b6a7914b2bc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.135807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 796/1582] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:28 2022] POST /identity/v3/users => generated 284 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:29.135807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb018c-144f-4c85-8198-a7015c2217e3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.136356 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971530ae-ebe8-4dff-aee2-b6a7914b2bc6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.145940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb018c-144f-4c85-8198-a7015c2217e3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.151644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52053f1b-e86b-4cc0-b972-abefd87f20fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.152707 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52053f1b-e86b-4cc0-b972-abefd87f20fa admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.152992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52053f1b-e86b-4cc0-b972-abefd87f20fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.153263 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52053f1b-e86b-4cc0-b972-abefd87f20fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.153988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52053f1b-e86b-4cc0-b972-abefd87f20fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.155642 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52053f1b-e86b-4cc0-b972-abefd87f20fa admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.170448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 797/1583] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:29.170448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52053f1b-e86b-4cc0-b972-abefd87f20fa admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.178386 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52053f1b-e86b-4cc0-b972-abefd87f20fa admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.183259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d5102-94fa-4c74-97c5-8ef569ef36f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.184139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d5102-94fa-4c74-97c5-8ef569ef36f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.184421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d5102-94fa-4c74-97c5-8ef569ef36f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.184697 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d5102-94fa-4c74-97c5-8ef569ef36f3 admin admin] PATH_INFO: `/v3/projects/c4807ce933f14af0be22dd1cf3635c2f/users/206ca9f278fc47e4a9a4f720083e91f3/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.199701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d5102-94fa-4c74-97c5-8ef569ef36f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4807ce933f14af0be22dd1cf3635c2f, user_id=206ca9f278fc47e4a9a4f720083e91f3, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.202403 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d5102-94fa-4c74-97c5-8ef569ef36f3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.217017 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 787/1584] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:29 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:29.217738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971530ae-ebe8-4dff-aee2-b6a7914b2bc6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.227692 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971530ae-ebe8-4dff-aee2-b6a7914b2bc6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.232251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b340ef5-a5d8-44c3-96e1-f758a72277f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.232881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b340ef5-a5d8-44c3-96e1-f758a72277f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.233162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b340ef5-a5d8-44c3-96e1-f758a72277f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.233434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b340ef5-a5d8-44c3-96e1-f758a72277f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.234133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b340ef5-a5d8-44c3-96e1-f758a72277f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.236087 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b340ef5-a5d8-44c3-96e1-f758a72277f1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.275448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 798/1585] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:29 2022] PUT /identity/v3/projects/c4807ce933f14af0be22dd1cf3635c2f/users/206ca9f278fc47e4a9a4f720083e91f3/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:29.279336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d5102-94fa-4c74-97c5-8ef569ef36f3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.349173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 788/1586] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:29 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:29.350329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d5102-94fa-4c74-97c5-8ef569ef36f3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.350951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b340ef5-a5d8-44c3-96e1-f758a72277f1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.355676 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bf13a-9fdd-47c7-b18f-20aef3c2b35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.356480 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bf13a-9fdd-47c7-b18f-20aef3c2b35f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.356869 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bf13a-9fdd-47c7-b18f-20aef3c2b35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.357224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bf13a-9fdd-47c7-b18f-20aef3c2b35f admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.357994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bf13a-9fdd-47c7-b18f-20aef3c2b35f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.359968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bf13a-9fdd-47c7-b18f-20aef3c2b35f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.362861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b340ef5-a5d8-44c3-96e1-f758a72277f1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.368609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece19b3-e3a4-440f-85d9-e72a93f3dd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.372626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece19b3-e3a4-440f-85d9-e72a93f3dd3d admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.372626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece19b3-e3a4-440f-85d9-e72a93f3dd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.372626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece19b3-e3a4-440f-85d9-e72a93f3dd3d admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.372626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece19b3-e3a4-440f-85d9-e72a93f3dd3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.372626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece19b3-e3a4-440f-85d9-e72a93f3dd3d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.429382 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 799/1587] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:29 2022] POST /identity/v3/users => generated 282 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:29.430256 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bf13a-9fdd-47c7-b18f-20aef3c2b35f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.439820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bf13a-9fdd-47c7-b18f-20aef3c2b35f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.441603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 789/1588] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:29 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:29.441603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece19b3-e3a4-440f-85d9-e72a93f3dd3d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.445019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0fb90-884a-40b1-b609-cf68d1b982f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.445678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0fb90-884a-40b1-b609-cf68d1b982f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.445962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0fb90-884a-40b1-b609-cf68d1b982f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.446242 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0fb90-884a-40b1-b609-cf68d1b982f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.446962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0fb90-884a-40b1-b609-cf68d1b982f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.448674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0fb90-884a-40b1-b609-cf68d1b982f9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.452882 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece19b3-e3a4-440f-85d9-e72a93f3dd3d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.457665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89911b20-4b4a-4c2a-9a16-915ac8759527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.458333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89911b20-4b4a-4c2a-9a16-915ac8759527 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.458618 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89911b20-4b4a-4c2a-9a16-915ac8759527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.458891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89911b20-4b4a-4c2a-9a16-915ac8759527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.458999 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 800/1589] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:29.459659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89911b20-4b4a-4c2a-9a16-915ac8759527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.459822 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0fb90-884a-40b1-b609-cf68d1b982f9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.461254 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89911b20-4b4a-4c2a-9a16-915ac8759527 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.471129 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0fb90-884a-40b1-b609-cf68d1b982f9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.472098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 790/1590] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:29.473374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e4697-070d-449a-8d0b-ba80bff86006 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.474310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e4697-070d-449a-8d0b-ba80bff86006 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.474641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e4697-070d-449a-8d0b-ba80bff86006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.482148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591092e-8b26-406c-adab-793c61d1c3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.482148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591092e-8b26-406c-adab-793c61d1c3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.482148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591092e-8b26-406c-adab-793c61d1c3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.482148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591092e-8b26-406c-adab-793c61d1c3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.482500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591092e-8b26-406c-adab-793c61d1c3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.483528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591092e-8b26-406c-adab-793c61d1c3f2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.493361 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8c4e4697-070d-449a-8d0b-ba80bff86006 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:29.494672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8c4e4697-070d-449a-8d0b-ba80bff86006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:29.498206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 801/1591] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:29.498206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591092e-8b26-406c-adab-793c61d1c3f2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.505458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 791/1592] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:29.510629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e4697-070d-449a-8d0b-ba80bff86006 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.510758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591092e-8b26-406c-adab-793c61d1c3f2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.516183 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e4697-070d-449a-8d0b-ba80bff86006 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.519830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1390336-b109-4277-9277-2444623d76dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.520595 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1390336-b109-4277-9277-2444623d76dd admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.520887 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1390336-b109-4277-9277-2444623d76dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.521221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1390336-b109-4277-9277-2444623d76dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.521339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7878e-09ec-4c5f-aec4-29f77c84f17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.522116 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7878e-09ec-4c5f-aec4-29f77c84f17b admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.522278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1390336-b109-4277-9277-2444623d76dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.522690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7878e-09ec-4c5f-aec4-29f77c84f17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.523043 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7878e-09ec-4c5f-aec4-29f77c84f17b admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.523892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7878e-09ec-4c5f-aec4-29f77c84f17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.524401 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1390336-b109-4277-9277-2444623d76dd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.525691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7878e-09ec-4c5f-aec4-29f77c84f17b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.544726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 802/1593] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:29.544929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 792/1594] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:29.545029 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1390336-b109-4277-9277-2444623d76dd admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.545121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7878e-09ec-4c5f-aec4-29f77c84f17b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.555553 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1390336-b109-4277-9277-2444623d76dd admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.556470 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7878e-09ec-4c5f-aec4-29f77c84f17b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.560373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66f9cb-31e3-449d-8db1-a6e8ee43ee4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.561119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66f9cb-31e3-449d-8db1-a6e8ee43ee4e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.561395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3919f-9444-4a3d-adca-a33deeeb8016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.561527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66f9cb-31e3-449d-8db1-a6e8ee43ee4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.561905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66f9cb-31e3-449d-8db1-a6e8ee43ee4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.562232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3919f-9444-4a3d-adca-a33deeeb8016 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.562596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3919f-9444-4a3d-adca-a33deeeb8016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.562859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66f9cb-31e3-449d-8db1-a6e8ee43ee4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.562997 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3919f-9444-4a3d-adca-a33deeeb8016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.563831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3919f-9444-4a3d-adca-a33deeeb8016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.564686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66f9cb-31e3-449d-8db1-a6e8ee43ee4e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.565704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3919f-9444-4a3d-adca-a33deeeb8016 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.578005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 803/1595] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:29.578005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66f9cb-31e3-449d-8db1-a6e8ee43ee4e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.578513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 793/1596] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:29.578599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3919f-9444-4a3d-adca-a33deeeb8016 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.590662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3919f-9444-4a3d-adca-a33deeeb8016 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.590891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66f9cb-31e3-449d-8db1-a6e8ee43ee4e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.595645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607c255-11f8-45f5-94dd-14e91d154606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.595815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca3c80-272e-42e5-aec4-96d5883ea66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.596446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca3c80-272e-42e5-aec4-96d5883ea66d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.596562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607c255-11f8-45f5-94dd-14e91d154606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.596815 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607c255-11f8-45f5-94dd-14e91d154606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.596935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca3c80-272e-42e5-aec4-96d5883ea66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.597101 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607c255-11f8-45f5-94dd-14e91d154606 admin admin] PATH_INFO: `/v3/projects/34f8d0620ab040d0b20215d0e40ecb97/users/d47c0721137343f2b5600ce5fd7de8d3/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.597355 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca3c80-272e-42e5-aec4-96d5883ea66d admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.599436 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca3c80-272e-42e5-aec4-96d5883ea66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.601837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:29.602148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca3c80-272e-42e5-aec4-96d5883ea66d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.613738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 794/1597] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:29.615451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca3c80-272e-42e5-aec4-96d5883ea66d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.618736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607c255-11f8-45f5-94dd-14e91d154606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f8d0620ab040d0b20215d0e40ecb97, user_id=d47c0721137343f2b5600ce5fd7de8d3, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.619092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5bca3c80-272e-42e5-aec4-96d5883ea66d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:29.621482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607c255-11f8-45f5-94dd-14e91d154606 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.670325 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 804/1598] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:29 2022] PUT /identity/v3/projects/34f8d0620ab040d0b20215d0e40ecb97/users/d47c0721137343f2b5600ce5fd7de8d3/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:29.671161 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607c255-11f8-45f5-94dd-14e91d154606 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.690707 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca3c80-272e-42e5-aec4-96d5883ea66d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.717370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:29.745805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c28984-0733-4603-8abe-9a6ec65903e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.746512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c28984-0733-4603-8abe-9a6ec65903e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.746805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c28984-0733-4603-8abe-9a6ec65903e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.747248 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c28984-0733-4603-8abe-9a6ec65903e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.758945 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607c255-11f8-45f5-94dd-14e91d154606 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.763972 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe02d6e-4b21-4518-86ad-6ab23ff9b937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.764718 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe02d6e-4b21-4518-86ad-6ab23ff9b937 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.765076 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe02d6e-4b21-4518-86ad-6ab23ff9b937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.765425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe02d6e-4b21-4518-86ad-6ab23ff9b937 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.766260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe02d6e-4b21-4518-86ad-6ab23ff9b937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.768017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe02d6e-4b21-4518-86ad-6ab23ff9b937 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.778753 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 805/1599] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:29.779675 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe02d6e-4b21-4518-86ad-6ab23ff9b937 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.790343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe02d6e-4b21-4518-86ad-6ab23ff9b937 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.796091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2e9f2-6cfd-4462-bc5e-deddb0ded1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.796756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2e9f2-6cfd-4462-bc5e-deddb0ded1ab admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.801350 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2e9f2-6cfd-4462-bc5e-deddb0ded1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.801350 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2e9f2-6cfd-4462-bc5e-deddb0ded1ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.801350 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2e9f2-6cfd-4462-bc5e-deddb0ded1ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.801350 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2e9f2-6cfd-4462-bc5e-deddb0ded1ab admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.815655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 806/1600] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:29.815822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c28984-0733-4603-8abe-9a6ec65903e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.815936 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2e9f2-6cfd-4462-bc5e-deddb0ded1ab admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.816050 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c28984-0733-4603-8abe-9a6ec65903e8 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.826513 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2e9f2-6cfd-4462-bc5e-deddb0ded1ab admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.826812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 795/1601] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:29.828204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c28984-0733-4603-8abe-9a6ec65903e8 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.831588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb244a-f6b0-4c93-a0b6-0d51cfb91c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.832393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb244a-f6b0-4c93-a0b6-0d51cfb91c66 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.832506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb244a-f6b0-4c93-a0b6-0d51cfb91c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.832793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb244a-f6b0-4c93-a0b6-0d51cfb91c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.834099 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb244a-f6b0-4c93-a0b6-0d51cfb91c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.837015 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb244a-f6b0-4c93-a0b6-0d51cfb91c66 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.841742 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c28984-0733-4603-8abe-9a6ec65903e8 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.847901 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e9922-374b-4cfd-8cac-a5bf858485ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.849594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e9922-374b-4cfd-8cac-a5bf858485ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.850953 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e9922-374b-4cfd-8cac-a5bf858485ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.851434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e9922-374b-4cfd-8cac-a5bf858485ee admin admin] PATH_INFO: `/v3/projects/6ea548217ba14fa4aabbfe81fa916275/users/32f1810ce2ae432685c4d7f642e43eb2/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.853191 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 807/1602] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:29.854082 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb244a-f6b0-4c93-a0b6-0d51cfb91c66 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:29.868359 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb244a-f6b0-4c93-a0b6-0d51cfb91c66 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:29.874039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f8ee7-f27c-4e3c-9769-cc0cdb4d541b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:29.874939 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f8ee7-f27c-4e3c-9769-cc0cdb4d541b admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.875297 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e9922-374b-4cfd-8cac-a5bf858485ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ea548217ba14fa4aabbfe81fa916275, user_id=32f1810ce2ae432685c4d7f642e43eb2, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.875429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f8ee7-f27c-4e3c-9769-cc0cdb4d541b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.875667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f8ee7-f27c-4e3c-9769-cc0cdb4d541b admin admin] PATH_INFO: `/v3/projects/b6bf0e1123fa47a7844e09127fb16d6e/users/b67761f9b13340a090efc1cb07714912/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.878379 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e9922-374b-4cfd-8cac-a5bf858485ee admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.892081 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f8ee7-f27c-4e3c-9769-cc0cdb4d541b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6bf0e1123fa47a7844e09127fb16d6e, user_id=b67761f9b13340a090efc1cb07714912, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:29.895091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f8ee7-f27c-4e3c-9769-cc0cdb4d541b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:29.949935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 796/1603] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:29 2022] PUT /identity/v3/projects/6ea548217ba14fa4aabbfe81fa916275/users/32f1810ce2ae432685c4d7f642e43eb2/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:29.951341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e97ae-e8c2-4742-8cf9-99160c91fd07 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:29.951627 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e97ae-e8c2-4742-8cf9-99160c91fd07 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:29.951902 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e97ae-e8c2-4742-8cf9-99160c91fd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:29.983384 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a57e97ae-e8c2-4742-8cf9-99160c91fd07 None None] MFA Rules not processed for user `d47c0721137343f2b5600ce5fd7de8d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:29.985199 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a57e97ae-e8c2-4742-8cf9-99160c91fd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:30.011374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 808/1604] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:29 2022] PUT /identity/v3/projects/b6bf0e1123fa47a7844e09127fb16d6e/users/b67761f9b13340a090efc1cb07714912/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:30.012454 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f8ee7-f27c-4e3c-9769-cc0cdb4d541b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:30.052335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 797/1605] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:29 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:30.053243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e97ae-e8c2-4742-8cf9-99160c91fd07 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:30.080579 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f8ee7-f27c-4e3c-9769-cc0cdb4d541b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:30.089982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55be13c-ade3-4b3d-bf70-dfd157e9ec6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:30.089982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55be13c-ade3-4b3d-bf70-dfd157e9ec6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:30.089982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55be13c-ade3-4b3d-bf70-dfd157e9ec6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:30.089982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55be13c-ade3-4b3d-bf70-dfd157e9ec6b admin admin] PATH_INFO: `/v3/projects/c6797657c8354a2c8eab88478857bd02/users/ac4b28c6fa2340aba557b7a81f38e84d/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:30.107235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:30.125439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e97ae-e8c2-4742-8cf9-99160c91fd07 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:30.126267 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55be13c-ade3-4b3d-bf70-dfd157e9ec6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6797657c8354a2c8eab88478857bd02, user_id=ac4b28c6fa2340aba557b7a81f38e84d, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:30.128977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55be13c-ade3-4b3d-bf70-dfd157e9ec6b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:30.131726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679470fa-9a92-420e-9629-4e850e99ed61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:30.132635 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679470fa-9a92-420e-9629-4e850e99ed61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:30.132903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679470fa-9a92-420e-9629-4e850e99ed61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:30.133194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679470fa-9a92-420e-9629-4e850e99ed61 admin admin] PATH_INFO: `/v3/projects/3326c29b85f643c28253c9ce948bda8a/users/a3cd1f48d64f47868ed582f79a7906f7/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:30.149462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679470fa-9a92-420e-9629-4e850e99ed61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3326c29b85f643c28253c9ce948bda8a, user_id=a3cd1f48d64f47868ed582f79a7906f7, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:30.152699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679470fa-9a92-420e-9629-4e850e99ed61 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:30.206478 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 809/1606] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:30 2022] PUT /identity/v3/projects/c6797657c8354a2c8eab88478857bd02/users/ac4b28c6fa2340aba557b7a81f38e84d/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:30.217473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55be13c-ade3-4b3d-bf70-dfd157e9ec6b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:30.232512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 798/1607] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:30 2022] PUT /identity/v3/projects/3326c29b85f643c28253c9ce948bda8a/users/a3cd1f48d64f47868ed582f79a7906f7/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:30.234443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef984a79-d40b-4e77-a680-06de2a119ce7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:30.235095 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef984a79-d40b-4e77-a680-06de2a119ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:30.235540 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef984a79-d40b-4e77-a680-06de2a119ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:30.241239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:30.270007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ef984a79-d40b-4e77-a680-06de2a119ce7 None None] MFA Rules not processed for user `32f1810ce2ae432685c4d7f642e43eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:30.278146 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ef984a79-d40b-4e77-a680-06de2a119ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:30.291993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55be13c-ade3-4b3d-bf70-dfd157e9ec6b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:30.363001 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 799/1608] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:30.365693 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1505d04-b9f4-4940-a0e4-e59a1dfb1538 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:30.365981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1505d04-b9f4-4940-a0e4-e59a1dfb1538 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:30.366262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1505d04-b9f4-4940-a0e4-e59a1dfb1538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:30.381117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b12f86-5e3b-4d4a-8679-6a33697d98fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:30.382039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b12f86-5e3b-4d4a-8679-6a33697d98fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:30.382322 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b12f86-5e3b-4d4a-8679-6a33697d98fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:30.382595 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b12f86-5e3b-4d4a-8679-6a33697d98fe admin admin] PATH_INFO: `/v3/projects/ebc1917e3470485696bfd6e97d67b5e5/users/860003a2d0e04181b812a5e72b3ffca9/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:30.401571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b12f86-5e3b-4d4a-8679-6a33697d98fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc1917e3470485696bfd6e97d67b5e5, user_id=860003a2d0e04181b812a5e72b3ffca9, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:30.404494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b12f86-5e3b-4d4a-8679-6a33697d98fe admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:30.405484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b1505d04-b9f4-4940-a0e4-e59a1dfb1538 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:30.407602 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b1505d04-b9f4-4940-a0e4-e59a1dfb1538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:30.417763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 800/1609] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:30.418323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1505d04-b9f4-4940-a0e4-e59a1dfb1538 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:30.433032 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1505d04-b9f4-4940-a0e4-e59a1dfb1538 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:30.438946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83b62a-3374-47b2-a107-fde9144b044a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:30.440054 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83b62a-3374-47b2-a107-fde9144b044a admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:30.441244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83b62a-3374-47b2-a107-fde9144b044a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:30.441596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83b62a-3374-47b2-a107-fde9144b044a admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:30.443133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83b62a-3374-47b2-a107-fde9144b044a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:30.445363 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83b62a-3374-47b2-a107-fde9144b044a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:30.480795 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 810/1610] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:30 2022] PUT /identity/v3/projects/ebc1917e3470485696bfd6e97d67b5e5/users/860003a2d0e04181b812a5e72b3ffca9/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:30.482109 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108259a-0d43-4499-aa75-cee2a76799a0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:30.482418 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108259a-0d43-4499-aa75-cee2a76799a0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:30.482691 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108259a-0d43-4499-aa75-cee2a76799a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:30.514227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c108259a-0d43-4499-aa75-cee2a76799a0 None None] MFA Rules not processed for user `ac4b28c6fa2340aba557b7a81f38e84d`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:30.515914 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c108259a-0d43-4499-aa75-cee2a76799a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:30.560612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 801/1611] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:30 2022] POST /identity/v3/projects => generated 369 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:30.562116 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4a8c9-d83e-4a32-a334-d8d998470f01 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:30.562451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4a8c9-d83e-4a32-a334-d8d998470f01 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:30.562779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4a8c9-d83e-4a32-a334-d8d998470f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:30.593317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-45e4a8c9-d83e-4a32-a334-d8d998470f01 None None] MFA Rules not processed for user `a3cd1f48d64f47868ed582f79a7906f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:30.596003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-45e4a8c9-d83e-4a32-a334-d8d998470f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:30.617103 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 811/1612] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:30 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:30.618397 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108259a-0d43-4499-aa75-cee2a76799a0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:30.701772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:30.716429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 802/1613] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:30.717297 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4a8c9-d83e-4a32-a334-d8d998470f01 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:30.721192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108259a-0d43-4499-aa75-cee2a76799a0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:30.739138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:30.781734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e229a-f060-4c8a-8cb3-0b223f1d3643 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:30.782724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e229a-f060-4c8a-8cb3-0b223f1d3643 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:30.783005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e229a-f060-4c8a-8cb3-0b223f1d3643 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:30.783464 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e229a-f060-4c8a-8cb3-0b223f1d3643 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:30.822548 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4a8c9-d83e-4a32-a334-d8d998470f01 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:30.830740 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93ad43-5cf2-440d-9a79-b80710d25a36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:30.831732 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93ad43-5cf2-440d-9a79-b80710d25a36 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:30.833414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93ad43-5cf2-440d-9a79-b80710d25a36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:30.833712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93ad43-5cf2-440d-9a79-b80710d25a36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:30.862343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e229a-f060-4c8a-8cb3-0b223f1d3643 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:30.864952 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e229a-f060-4c8a-8cb3-0b223f1d3643 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:30.876940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 812/1614] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:30.878987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cf54a-1b3c-4c78-8039-4ef62f05b306 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:30.879665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cf54a-1b3c-4c78-8039-4ef62f05b306 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:30.881038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cf54a-1b3c-4c78-8039-4ef62f05b306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:30.919631 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-045cf54a-1b3c-4c78-8039-4ef62f05b306 None None] MFA Rules not processed for user `860003a2d0e04181b812a5e72b3ffca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:30.920925 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-045cf54a-1b3c-4c78-8039-4ef62f05b306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:30.937142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93ad43-5cf2-440d-9a79-b80710d25a36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:30.940130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93ad43-5cf2-440d-9a79-b80710d25a36 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:30.959362 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 803/1615] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:30.960305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93ad43-5cf2-440d-9a79-b80710d25a36 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:30.981193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93ad43-5cf2-440d-9a79-b80710d25a36 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:30.997451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b6572-fca6-4bea-b7c8-69ed910af396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:30.998151 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b6572-fca6-4bea-b7c8-69ed910af396 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:30.998848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b6572-fca6-4bea-b7c8-69ed910af396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:30.999131 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b6572-fca6-4bea-b7c8-69ed910af396 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:30.999869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b6572-fca6-4bea-b7c8-69ed910af396 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:31.002320 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b6572-fca6-4bea-b7c8-69ed910af396 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:31.021250 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 813/1616] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:31.023734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cf54a-1b3c-4c78-8039-4ef62f05b306 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:31.037492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cf54a-1b3c-4c78-8039-4ef62f05b306 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:31.042962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51002b9d-0f2e-4998-ab01-44ffea888f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:31.043681 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51002b9d-0f2e-4998-ab01-44ffea888f81 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:31.043982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51002b9d-0f2e-4998-ab01-44ffea888f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:31.044282 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51002b9d-0f2e-4998-ab01-44ffea888f81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:31.044980 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51002b9d-0f2e-4998-ab01-44ffea888f81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:31.047002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51002b9d-0f2e-4998-ab01-44ffea888f81 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:31.141987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 804/1617] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:30 2022] POST /identity/v3/users => generated 291 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:31.142473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cdb1d-e7f0-4f0b-9a3a-d07c2713d69f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:31.142473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cdb1d-e7f0-4f0b-9a3a-d07c2713d69f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:31.142723 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cdb1d-e7f0-4f0b-9a3a-d07c2713d69f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:31.146654 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 805/1618] 10.222.0.67 () {54 vars in 961 bytes} [Fri Jun 24 15:18:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:18:31.147754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32cdb1d-e7f0-4f0b-9a3a-d07c2713d69f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:31.175031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 814/1619] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:31 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:31.178236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51002b9d-0f2e-4998-ab01-44ffea888f81 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:31.205349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32cdb1d-e7f0-4f0b-9a3a-d07c2713d69f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:31.240470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:31.321684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51002b9d-0f2e-4998-ab01-44ffea888f81 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:31.346090 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4778b-81c6-409f-93fb-6c182c322d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:31.347101 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4778b-81c6-409f-93fb-6c182c322d3e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:31.347747 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4778b-81c6-409f-93fb-6c182c322d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:31.348185 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4778b-81c6-409f-93fb-6c182c322d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:31.349251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4778b-81c6-409f-93fb-6c182c322d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:31.351068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4778b-81c6-409f-93fb-6c182c322d3e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:31.375524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 815/1620] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:31.378655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78f21d-3e0a-401c-9647-a4c3850cfd3a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:31.379196 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78f21d-3e0a-401c-9647-a4c3850cfd3a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:31.380430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78f21d-3e0a-401c-9647-a4c3850cfd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:31.397275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c660c86-a07d-42ae-b499-2e7c83c76ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:31.397992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c660c86-a07d-42ae-b499-2e7c83c76ade admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:31.398345 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c660c86-a07d-42ae-b499-2e7c83c76ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:31.398688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c660c86-a07d-42ae-b499-2e7c83c76ade admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:31.399533 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c660c86-a07d-42ae-b499-2e7c83c76ade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:31.402034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c660c86-a07d-42ae-b499-2e7c83c76ade admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:31.419311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-db78f21d-3e0a-401c-9647-a4c3850cfd3a None None] MFA Rules not processed for user `51de59caa3144a1cbf12b20b15e2134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:31.421418 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-db78f21d-3e0a-401c-9647-a4c3850cfd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:31.606066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 806/1621] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:31 2022] POST /identity/v3/projects => generated 381 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:31.606066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c660c86-a07d-42ae-b499-2e7c83c76ade admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:31.626982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 816/1622] 10.222.0.67 () {58 vars in 1058 bytes} [Fri Jun 24 15:18:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:31.627904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78f21d-3e0a-401c-9647-a4c3850cfd3a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:31.648413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78f21d-3e0a-401c-9647-a4c3850cfd3a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:31.678760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:31.738549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c660c86-a07d-42ae-b499-2e7c83c76ade admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:31.755621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3c664-0179-42a9-a300-962747b3a104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:31.755621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3c664-0179-42a9-a300-962747b3a104 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:31.755621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3c664-0179-42a9-a300-962747b3a104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:31.755621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3c664-0179-42a9-a300-962747b3a104 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:31.756606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3c664-0179-42a9-a300-962747b3a104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:31.757435 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3c664-0179-42a9-a300-962747b3a104 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:31.782736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a84db4-aeba-4f29-8e36-a9159219ca01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:31.782736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a84db4-aeba-4f29-8e36-a9159219ca01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:31.782736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a84db4-aeba-4f29-8e36-a9159219ca01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:31.782736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a84db4-aeba-4f29-8e36-a9159219ca01 admin admin] PATH_INFO: `/v3/projects/39ce535da8614cc8a423316e7077eace/users/daa83908246c44e992fe7906de362c71/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:31.800493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a84db4-aeba-4f29-8e36-a9159219ca01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ce535da8614cc8a423316e7077eace, user_id=daa83908246c44e992fe7906de362c71, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:31.820455 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a84db4-aeba-4f29-8e36-a9159219ca01 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:31.872357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 807/1623] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:31 2022] POST /identity/v3/users => generated 289 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:31.873445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3c664-0179-42a9-a300-962747b3a104 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:31.884661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3c664-0179-42a9-a300-962747b3a104 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:31.890854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affa928-3650-443f-ba67-19b21e5a6833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:31.891646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affa928-3650-443f-ba67-19b21e5a6833 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:31.891938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affa928-3650-443f-ba67-19b21e5a6833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:31.892227 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affa928-3650-443f-ba67-19b21e5a6833 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:31.893227 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affa928-3650-443f-ba67-19b21e5a6833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:31.895121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affa928-3650-443f-ba67-19b21e5a6833 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:31.901025 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 817/1624] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:31 2022] PUT /identity/v3/projects/39ce535da8614cc8a423316e7077eace/users/daa83908246c44e992fe7906de362c71/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:31.902445 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a84db4-aeba-4f29-8e36-a9159219ca01 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:31.990231 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 808/1625] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:31 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:31.994746 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affa928-3650-443f-ba67-19b21e5a6833 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:32.005693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a84db4-aeba-4f29-8e36-a9159219ca01 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:32.012212 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affa928-3650-443f-ba67-19b21e5a6833 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:32.012977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5504c7-32fe-43cd-b34f-92fcff6b7e16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:32.013735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5504c7-32fe-43cd-b34f-92fcff6b7e16 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:32.014858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5504c7-32fe-43cd-b34f-92fcff6b7e16 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:32.014858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5504c7-32fe-43cd-b34f-92fcff6b7e16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:32.079917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:32.107698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5504c7-32fe-43cd-b34f-92fcff6b7e16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:32.109551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5504c7-32fe-43cd-b34f-92fcff6b7e16 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:32.124374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 818/1626] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:32.125569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5504c7-32fe-43cd-b34f-92fcff6b7e16 service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:32.161820 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afec33-64c5-4bcf-850b-4b1aa263b1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:32.163473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afec33-64c5-4bcf-850b-4b1aa263b1a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:32.163754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afec33-64c5-4bcf-850b-4b1aa263b1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:32.164195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afec33-64c5-4bcf-850b-4b1aa263b1a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:32.164755 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afec33-64c5-4bcf-850b-4b1aa263b1a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:32.194408 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afec33-64c5-4bcf-850b-4b1aa263b1a8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:32.231645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 809/1627] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:32.235604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afec33-64c5-4bcf-850b-4b1aa263b1a8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:32.236875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5504c7-32fe-43cd-b34f-92fcff6b7e16 service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:32.242759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a066d5f-be3c-4146-9502-a01c349a05c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:32.244334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a066d5f-be3c-4146-9502-a01c349a05c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:32.244334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a066d5f-be3c-4146-9502-a01c349a05c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:32.244334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a066d5f-be3c-4146-9502-a01c349a05c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:32.244923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a066d5f-be3c-4146-9502-a01c349a05c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:32.246607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a066d5f-be3c-4146-9502-a01c349a05c8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:32.262253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 819/1628] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:32.262253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a066d5f-be3c-4146-9502-a01c349a05c8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:32.264177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afec33-64c5-4bcf-850b-4b1aa263b1a8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:32.276566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a066d5f-be3c-4146-9502-a01c349a05c8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:32.277462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59299c4-4bbc-4fbd-8442-005c29c19a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:32.278607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59299c4-4bbc-4fbd-8442-005c29c19a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:32.279166 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59299c4-4bbc-4fbd-8442-005c29c19a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:32.279889 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59299c4-4bbc-4fbd-8442-005c29c19a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:32.280792 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59299c4-4bbc-4fbd-8442-005c29c19a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:32.282513 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b14bb-8350-4722-995f-617685c75dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:32.284849 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59299c4-4bbc-4fbd-8442-005c29c19a20 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:32.285267 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b14bb-8350-4722-995f-617685c75dd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:32.285556 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b14bb-8350-4722-995f-617685c75dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:32.285834 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b14bb-8350-4722-995f-617685c75dd3 admin admin] PATH_INFO: `/v3/projects/78bb138ba51f4372ad082b535e2160cb/users/918ac24195b84bd599d66e29e960bcca/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:32.316849 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 810/1629] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:32.320680 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59299c4-4bbc-4fbd-8442-005c29c19a20 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:32.343038 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59299c4-4bbc-4fbd-8442-005c29c19a20 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:32.348767 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b14bb-8350-4722-995f-617685c75dd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78bb138ba51f4372ad082b535e2160cb, user_id=918ac24195b84bd599d66e29e960bcca, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:32.354124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b14bb-8350-4722-995f-617685c75dd3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:32.356016 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9748282-108a-483b-b089-4f3592af3c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:32.357109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9748282-108a-483b-b089-4f3592af3c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:32.357572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9748282-108a-483b-b089-4f3592af3c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:32.358004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9748282-108a-483b-b089-4f3592af3c9d admin admin] PATH_INFO: `/v3/projects/39ce535da8614cc8a423316e7077eace/users/daa83908246c44e992fe7906de362c71/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:32.361785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9748282-108a-483b-b089-4f3592af3c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ce535da8614cc8a423316e7077eace, user_id=daa83908246c44e992fe7906de362c71, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:32.366384 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9748282-108a-483b-b089-4f3592af3c9d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:32.402246 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 811/1630] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:32 2022] PUT /identity/v3/projects/39ce535da8614cc8a423316e7077eace/users/daa83908246c44e992fe7906de362c71/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:32.417458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9748282-108a-483b-b089-4f3592af3c9d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:32.452490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 820/1631] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:32 2022] PUT /identity/v3/projects/78bb138ba51f4372ad082b535e2160cb/users/918ac24195b84bd599d66e29e960bcca/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:32.454731 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e61406-11ce-4368-9f36-a145c613b169 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:32.455206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e61406-11ce-4368-9f36-a145c613b169 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:32.460550 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e61406-11ce-4368-9f36-a145c613b169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:32.538624 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-53e61406-11ce-4368-9f36-a145c613b169 None None] MFA Rules not processed for user `daa83908246c44e992fe7906de362c71`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:32.543229 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-53e61406-11ce-4368-9f36-a145c613b169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:32.602864 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9748282-108a-483b-b089-4f3592af3c9d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:32.663725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:32.707262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 821/1632] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:32 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:32.708439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e61406-11ce-4368-9f36-a145c613b169 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:32.766939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac5daa-6958-48db-99df-85d12cfcc593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:32.768491 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac5daa-6958-48db-99df-85d12cfcc593 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:32.768780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac5daa-6958-48db-99df-85d12cfcc593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:32.769066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac5daa-6958-48db-99df-85d12cfcc593 admin admin] PATH_INFO: `/v3/projects/a207aa7fd3764fe1b8741958d32c8bc6/users/7946e39bcfa54daea7459a3173965317/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:32.801022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac5daa-6958-48db-99df-85d12cfcc593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a207aa7fd3764fe1b8741958d32c8bc6, user_id=7946e39bcfa54daea7459a3173965317, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:32.804734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac5daa-6958-48db-99df-85d12cfcc593 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:32.824349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e61406-11ce-4368-9f36-a145c613b169 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:32.830968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53ddb6-9d5c-4003-878f-0dfe92df1bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:32.832582 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53ddb6-9d5c-4003-878f-0dfe92df1bac admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:32.840996 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53ddb6-9d5c-4003-878f-0dfe92df1bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:32.840996 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53ddb6-9d5c-4003-878f-0dfe92df1bac admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:32.840996 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53ddb6-9d5c-4003-878f-0dfe92df1bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:32.840996 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53ddb6-9d5c-4003-878f-0dfe92df1bac admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:32.851382 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 822/1633] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:32.853434 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53ddb6-9d5c-4003-878f-0dfe92df1bac admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:32.888879 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 812/1634] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:32 2022] PUT /identity/v3/projects/a207aa7fd3764fe1b8741958d32c8bc6/users/7946e39bcfa54daea7459a3173965317/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:32.890749 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2de0f-072b-4534-b67d-0780049dab56 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:32.891446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2de0f-072b-4534-b67d-0780049dab56 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:32.891746 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2de0f-072b-4534-b67d-0780049dab56 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:32.896008 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 813/1635] 10.222.0.67 () {54 vars in 912 bytes} [Fri Jun 24 15:18:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:18:32.896836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2de0f-072b-4534-b67d-0780049dab56 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:32.926363 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53ddb6-9d5c-4003-878f-0dfe92df1bac admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:33.002288 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad32dd-315f-4e6e-9985-564670a6f4f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:33.003519 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad32dd-315f-4e6e-9985-564670a6f4f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:33.003814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad32dd-315f-4e6e-9985-564670a6f4f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:33.005431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad32dd-315f-4e6e-9985-564670a6f4f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:33.012938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2de0f-072b-4534-b67d-0780049dab56 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:33.024811 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbdb2b-7b08-4ba7-b87b-7d7be4270f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:33.027563 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbdb2b-7b08-4ba7-b87b-7d7be4270f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:33.027932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbdb2b-7b08-4ba7-b87b-7d7be4270f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:33.028283 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbdb2b-7b08-4ba7-b87b-7d7be4270f68 admin admin] PATH_INFO: `/v3/projects/78bb138ba51f4372ad082b535e2160cb/users/918ac24195b84bd599d66e29e960bcca/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:33.038422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbdb2b-7b08-4ba7-b87b-7d7be4270f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78bb138ba51f4372ad082b535e2160cb, user_id=918ac24195b84bd599d66e29e960bcca, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:33.039896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbdb2b-7b08-4ba7-b87b-7d7be4270f68 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:33.074862 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 814/1636] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:32 2022] PUT /identity/v3/projects/78bb138ba51f4372ad082b535e2160cb/users/918ac24195b84bd599d66e29e960bcca/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:33.076263 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbdb2b-7b08-4ba7-b87b-7d7be4270f68 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:33.094748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad32dd-315f-4e6e-9985-564670a6f4f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:33.096663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad32dd-315f-4e6e-9985-564670a6f4f9 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:33.113233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 823/1637] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:32 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:33.115944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9352f-95a6-46d7-8b3b-06c14fe52d59 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:33.116461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9352f-95a6-46d7-8b3b-06c14fe52d59 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:33.116936 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9352f-95a6-46d7-8b3b-06c14fe52d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:33.132197 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:33.152525 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-87c9352f-95a6-46d7-8b3b-06c14fe52d59 None None] MFA Rules not processed for user `51de59caa3144a1cbf12b20b15e2134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:33.154987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-87c9352f-95a6-46d7-8b3b-06c14fe52d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:33.203228 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbdb2b-7b08-4ba7-b87b-7d7be4270f68 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:33.214214 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851458b-acf9-4531-9f04-607ee3ee9d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:33.215614 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851458b-acf9-4531-9f04-607ee3ee9d2d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:33.216384 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851458b-acf9-4531-9f04-607ee3ee9d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:33.216949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851458b-acf9-4531-9f04-607ee3ee9d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:33.217878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851458b-acf9-4531-9f04-607ee3ee9d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:33.220069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851458b-acf9-4531-9f04-607ee3ee9d2d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:33.258876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 815/1638] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:33.258876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bfc46-525f-498a-b49e-5c018d0b811c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:33.258876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bfc46-525f-498a-b49e-5c018d0b811c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:33.258876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bfc46-525f-498a-b49e-5c018d0b811c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:33.306057 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 824/1639] 10.222.0.67 () {58 vars in 1009 bytes} [Fri Jun 24 15:18:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:33.309308 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9352f-95a6-46d7-8b3b-06c14fe52d59 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:33.331288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-890bfc46-525f-498a-b49e-5c018d0b811c None None] MFA Rules not processed for user `918ac24195b84bd599d66e29e960bcca`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:33.334927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-890bfc46-525f-498a-b49e-5c018d0b811c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:33.459336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9352f-95a6-46d7-8b3b-06c14fe52d59 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:33.468250 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b49129-b1e7-460e-8416-5df34dd763c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:33.469616 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b49129-b1e7-460e-8416-5df34dd763c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:33.470902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b49129-b1e7-460e-8416-5df34dd763c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:33.472857 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b49129-b1e7-460e-8416-5df34dd763c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:33.511296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 816/1640] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:33.512434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bfc46-525f-498a-b49e-5c018d0b811c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:33.527215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bfc46-525f-498a-b49e-5c018d0b811c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:33.533922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab2e61-1441-4815-b013-c2d182f721c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:33.534929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab2e61-1441-4815-b013-c2d182f721c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:33.535306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab2e61-1441-4815-b013-c2d182f721c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:33.535648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab2e61-1441-4815-b013-c2d182f721c4 admin admin] PATH_INFO: `/v3/projects/a207aa7fd3764fe1b8741958d32c8bc6/users/7946e39bcfa54daea7459a3173965317/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:33.538926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab2e61-1441-4815-b013-c2d182f721c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a207aa7fd3764fe1b8741958d32c8bc6, user_id=7946e39bcfa54daea7459a3173965317, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:33.541966 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab2e61-1441-4815-b013-c2d182f721c4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:33.564332 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 817/1641] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:33 2022] PUT /identity/v3/projects/a207aa7fd3764fe1b8741958d32c8bc6/users/7946e39bcfa54daea7459a3173965317/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:33.564332 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab2e61-1441-4815-b013-c2d182f721c4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:33.590758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b49129-b1e7-460e-8416-5df34dd763c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:33.592689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b49129-b1e7-460e-8416-5df34dd763c2 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:33.608941 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 825/1642] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:33 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:33.611213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559308d1-43f4-4150-a847-f0fa6572cde5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:33.612050 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559308d1-43f4-4150-a847-f0fa6572cde5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:33.612358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559308d1-43f4-4150-a847-f0fa6572cde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:33.658516 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-559308d1-43f4-4150-a847-f0fa6572cde5 None None] MFA Rules not processed for user `7946e39bcfa54daea7459a3173965317`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:33.661479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-559308d1-43f4-4150-a847-f0fa6572cde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:33.721505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab2e61-1441-4815-b013-c2d182f721c4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:33.732743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed7e01-87a1-4986-9aee-9283aa8710da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:33.735055 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed7e01-87a1-4986-9aee-9283aa8710da service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:33.735498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed7e01-87a1-4986-9aee-9283aa8710da service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:33.736422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed7e01-87a1-4986-9aee-9283aa8710da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:33.772829 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:33.788421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 826/1643] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:33.891047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed7e01-87a1-4986-9aee-9283aa8710da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:33.893172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed7e01-87a1-4986-9aee-9283aa8710da service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:33.919666 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 818/1644] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:34.171722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:35.161839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:35.161839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559308d1-43f4-4150-a847-f0fa6572cde5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:35.161839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559308d1-43f4-4150-a847-f0fa6572cde5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:35.161839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616dc9b9-0c2d-4529-a3fe-3a64663d71af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:35.165899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dc9b9-0c2d-4529-a3fe-3a64663d71af service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:35.165899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dc9b9-0c2d-4529-a3fe-3a64663d71af service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:35.165899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dc9b9-0c2d-4529-a3fe-3a64663d71af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:35.169894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed7e01-87a1-4986-9aee-9283aa8710da service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:35.169894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed7e01-87a1-4986-9aee-9283aa8710da service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:35.169894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49b255-8fb1-49a5-8079-118d97b37b40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:35.169894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49b255-8fb1-49a5-8079-118d97b37b40 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:35.173725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49b255-8fb1-49a5-8079-118d97b37b40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:35.173725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49b255-8fb1-49a5-8079-118d97b37b40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:35.173725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49b255-8fb1-49a5-8079-118d97b37b40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:35.173725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49b255-8fb1-49a5-8079-118d97b37b40 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:35.173725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 819/1645] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:18:35.235234 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616dc9b9-0c2d-4529-a3fe-3a64663d71af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:35.240225 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616dc9b9-0c2d-4529-a3fe-3a64663d71af service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:35.284330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 827/1646] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:34 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:35.709619 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49b255-8fb1-49a5-8079-118d97b37b40 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:35.718507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49b255-8fb1-49a5-8079-118d97b37b40 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:35.732957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3331e40-7bba-4453-accd-fd903c57e826 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:35.733877 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3331e40-7bba-4453-accd-fd903c57e826 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:35.734063 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3331e40-7bba-4453-accd-fd903c57e826 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:35.743378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3331e40-7bba-4453-accd-fd903c57e826 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:35.757746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd017e-fc15-423f-8598-3dd910d432c1 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:35.758072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd017e-fc15-423f-8598-3dd910d432c1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:35.758304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd017e-fc15-423f-8598-3dd910d432c1 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:35.760943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 828/1647] 10.222.0.67 () {54 vars in 961 bytes} [Fri Jun 24 15:18:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:18:35.785382 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143e0b2-6921-4b83-a061-8e395a425edb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:35.785677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143e0b2-6921-4b83-a061-8e395a425edb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:35.786601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143e0b2-6921-4b83-a061-8e395a425edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:35.828341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d143e0b2-6921-4b83-a061-8e395a425edb None None] MFA Rules not processed for user `51de59caa3144a1cbf12b20b15e2134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:35.829512 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d143e0b2-6921-4b83-a061-8e395a425edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:35.848811 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 829/1648] 10.222.0.67 () {58 vars in 1058 bytes} [Fri Jun 24 15:18:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:18:35.850559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136d3c0-25cd-478e-9862-c12946f4590f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:35.850858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136d3c0-25cd-478e-9862-c12946f4590f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:35.851137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136d3c0-25cd-478e-9862-c12946f4590f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:35.858088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 830/1649] 10.222.0.67 () {54 vars in 912 bytes} [Fri Jun 24 15:18:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:18:35.859522 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c28a2f-fa82-4000-87dc-680ff2c30b45 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:35.859813 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c28a2f-fa82-4000-87dc-680ff2c30b45 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:35.860095 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c28a2f-fa82-4000-87dc-680ff2c30b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:35.886376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:35.886549 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d6c28a2f-fa82-4000-87dc-680ff2c30b45 None None] MFA Rules not processed for user `51de59caa3144a1cbf12b20b15e2134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:35.888599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d6c28a2f-fa82-4000-87dc-680ff2c30b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:35.919878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 831/1650] 10.222.0.67 () {58 vars in 1009 bytes} [Fri Jun 24 15:18:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:35.937132 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3331e40-7bba-4453-accd-fd903c57e826 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:35.938411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3331e40-7bba-4453-accd-fd903c57e826 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:35.956892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 820/1651] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:36.076949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c28a2f-fa82-4000-87dc-680ff2c30b45 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:36.107686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c28a2f-fa82-4000-87dc-680ff2c30b45 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:36.123201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48116f0-ff7c-4883-90f4-5280b3bf3f7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:36.124159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48116f0-ff7c-4883-90f4-5280b3bf3f7f service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:36.125817 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48116f0-ff7c-4883-90f4-5280b3bf3f7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:36.126859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48116f0-ff7c-4883-90f4-5280b3bf3f7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:36.149686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48116f0-ff7c-4883-90f4-5280b3bf3f7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:36.151915 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48116f0-ff7c-4883-90f4-5280b3bf3f7f service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:36.170853 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 832/1652] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:39.047621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1c41d-9465-4b8e-8cb4-6aa079651c38 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:39.047621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1c41d-9465-4b8e-8cb4-6aa079651c38 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:39.048277 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1c41d-9465-4b8e-8cb4-6aa079651c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:39.103679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c0d1c41d-9465-4b8e-8cb4-6aa079651c38 None None] MFA Rules not processed for user `78195ff60f0242ada974a8e08d1f2754`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:39.104922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c0d1c41d-9465-4b8e-8cb4-6aa079651c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:39.237960 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 821/1653] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:39.252201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13cfe9-dcec-4314-9b25-e81db033d67d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:39.252726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13cfe9-dcec-4314-9b25-e81db033d67d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:39.252805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13cfe9-dcec-4314-9b25-e81db033d67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:39.285457 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2b13cfe9-dcec-4314-9b25-e81db033d67d None None] MFA Rules not processed for user `78195ff60f0242ada974a8e08d1f2754`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:39.287248 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2b13cfe9-dcec-4314-9b25-e81db033d67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:39.343117 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1c41d-9465-4b8e-8cb4-6aa079651c38 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:39.353329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 833/1654] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:39.365711 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1c41d-9465-4b8e-8cb4-6aa079651c38 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:39.390000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30614d92-6745-49b9-9b50-d9675ac7aa26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:39.405960 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30614d92-6745-49b9-9b50-d9675ac7aa26 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:39.406530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30614d92-6745-49b9-9b50-d9675ac7aa26 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:39.407483 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30614d92-6745-49b9-9b50-d9675ac7aa26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:39.434948 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30614d92-6745-49b9-9b50-d9675ac7aa26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:39.445146 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30614d92-6745-49b9-9b50-d9675ac7aa26 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:39.482807 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 822/1655] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:18:39.883206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13cfe9-dcec-4314-9b25-e81db033d67d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:40.038164 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:40.050932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13cfe9-dcec-4314-9b25-e81db033d67d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:40.060364 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d6d2e-79d2-4f32-bd0c-bdef448e3ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:40.061984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d6d2e-79d2-4f32-bd0c-bdef448e3ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:40.063156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d6d2e-79d2-4f32-bd0c-bdef448e3ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:40.063734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d6d2e-79d2-4f32-bd0c-bdef448e3ab5 admin admin] PATH_INFO: `/v3/users/ac4b28c6fa2340aba557b7a81f38e84d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:40.065461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d6d2e-79d2-4f32-bd0c-bdef448e3ab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac4b28c6fa2340aba557b7a81f38e84d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:40.074336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d6d2e-79d2-4f32-bd0c-bdef448e3ab5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:40.240094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bfc4b-8b65-44e9-8c20-ea81d3436563 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:40.240616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bfc4b-8b65-44e9-8c20-ea81d3436563 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:40.240849 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bfc4b-8b65-44e9-8c20-ea81d3436563 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:40.256034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 823/1656] 10.222.0.67 () {54 vars in 912 bytes} [Fri Jun 24 15:18:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:18:40.256034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd78d3-0288-461b-9637-6e70e0fd1c29 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:40.256034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd78d3-0288-461b-9637-6e70e0fd1c29 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:40.256034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd78d3-0288-461b-9637-6e70e0fd1c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:40.296125 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-838d6d2e-79d2-4f32-bd0c-bdef448e3ab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac4b28c6fa2340aba557b7a81f38e84d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:40.303927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-72dd78d3-0288-461b-9637-6e70e0fd1c29 None None] MFA Rules not processed for user `95e7fb8f48dd4348b54e3694a8410bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:40.306725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-72dd78d3-0288-461b-9637-6e70e0fd1c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:40.332576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-838d6d2e-79d2-4f32-bd0c-bdef448e3ab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac4b28c6fa2340aba557b7a81f38e84d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:40.333365 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-838d6d2e-79d2-4f32-bd0c-bdef448e3ab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac4b28c6fa2340aba557b7a81f38e84d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:40.348347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-838d6d2e-79d2-4f32-bd0c-bdef448e3ab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac4b28c6fa2340aba557b7a81f38e84d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:40.350350 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-838d6d2e-79d2-4f32-bd0c-bdef448e3ab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac4b28c6fa2340aba557b7a81f38e84d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:40.402205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 824/1657] 10.222.0.67 () {58 vars in 1009 bytes} [Fri Jun 24 15:18:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:18:40.408345 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd78d3-0288-461b-9637-6e70e0fd1c29 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:40.411896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-72dd78d3-0288-461b-9637-6e70e0fd1c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:40.426067 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd78d3-0288-461b-9637-6e70e0fd1c29 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:40.443332 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa269d2-b849-4cf7-8817-313007f92512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:40.447466 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa269d2-b849-4cf7-8817-313007f92512 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:40.448789 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa269d2-b849-4cf7-8817-313007f92512 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:40.449234 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa269d2-b849-4cf7-8817-313007f92512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:40.472830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 834/1658] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:18:39 2022] DELETE /identity/v3/users/ac4b28c6fa2340aba557b7a81f38e84d => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:40.481200 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26268b0-ec5f-49e3-ac07-80301ba6f136 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:40.481753 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26268b0-ec5f-49e3-ac07-80301ba6f136 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:40.482168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26268b0-ec5f-49e3-ac07-80301ba6f136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:40.489614 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa269d2-b849-4cf7-8817-313007f92512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:40.491479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa269d2-b849-4cf7-8817-313007f92512 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:40.508596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 825/1659] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:40.537871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b26268b0-ec5f-49e3-ac07-80301ba6f136 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:40.549113 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b26268b0-ec5f-49e3-ac07-80301ba6f136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:41.067412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 835/1660] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 587 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:41.072540 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa269d2-b849-4cf7-8817-313007f92512 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:41.088672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa269d2-b849-4cf7-8817-313007f92512 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:41.148979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:41.196516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ffdd5-efc2-4f00-9705-5d6b9001f766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:41.197206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ffdd5-efc2-4f00-9705-5d6b9001f766 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:41.197488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ffdd5-efc2-4f00-9705-5d6b9001f766 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:41.199056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ffdd5-efc2-4f00-9705-5d6b9001f766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:41.227006 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ffdd5-efc2-4f00-9705-5d6b9001f766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:41.228762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ffdd5-efc2-4f00-9705-5d6b9001f766 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:41.251303 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 826/1661] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:41.627320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedba11f-3fb1-48cc-bc88-f006f38b6bfc None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:41.632437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedba11f-3fb1-48cc-bc88-f006f38b6bfc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:41.632437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedba11f-3fb1-48cc-bc88-f006f38b6bfc None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:41.632437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 836/1662] 10.222.0.67 () {54 vars in 960 bytes} [Fri Jun 24 15:18:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:18:41.638109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99c732-1a6d-48fe-b845-a8594fce6dc2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:41.638775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99c732-1a6d-48fe-b845-a8594fce6dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:41.639153 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99c732-1a6d-48fe-b845-a8594fce6dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:41.694337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8b99c732-1a6d-48fe-b845-a8594fce6dc2 None None] MFA Rules not processed for user `f612d6ff06fc458e8c1178cbf8321f79`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:41.697972 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8b99c732-1a6d-48fe-b845-a8594fce6dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:41.932190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 827/1663] 10.222.0.67 () {58 vars in 1057 bytes} [Fri Jun 24 15:18:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:41.963367 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedba11f-3fb1-48cc-bc88-f006f38b6bfc None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:41.985630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedba11f-3fb1-48cc-bc88-f006f38b6bfc None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:41.992737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03224853-0637-43e9-b6d3-f1d74aa430b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:42.002199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03224853-0637-43e9-b6d3-f1d74aa430b5 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:42.002199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03224853-0637-43e9-b6d3-f1d74aa430b5 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:42.002199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03224853-0637-43e9-b6d3-f1d74aa430b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:42.002199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-03224853-0637-43e9-b6d3-f1d74aa430b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:42.018899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03224853-0637-43e9-b6d3-f1d74aa430b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:42.025499 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03224853-0637-43e9-b6d3-f1d74aa430b5 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:42.042774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99c732-1a6d-48fe-b845-a8594fce6dc2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:42.044816 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8b99c732-1a6d-48fe-b845-a8594fce6dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:42.065600 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 837/1664] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:42.103155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99c732-1a6d-48fe-b845-a8594fce6dc2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:42.119964 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:42.123163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:42.123744 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:42.124211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6 admin admin] PATH_INFO: `/v3/projects/c6797657c8354a2c8eab88478857bd02` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:42.127829 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6797657c8354a2c8eab88478857bd02)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:42.134326 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:42.338232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6797657c8354a2c8eab88478857bd02', 'request_id': 'req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:42.352974 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6797657c8354a2c8eab88478857bd02', 'request_id': 'req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:42.494174 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6797657c8354a2c8eab88478857bd02', 'request_id': 'req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:42.512897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6 admin admin] The token cache is being invalidate because project c6797657c8354a2c8eab88478857bd02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:18:42.513419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:42.514380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:42.517311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 828/1665] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:18:42 2022] DELETE /identity/v3/projects/c6797657c8354a2c8eab88478857bd02 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:42.523914 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03224853-0637-43e9-b6d3-f1d74aa430b5 service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:42.563098 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03224853-0637-43e9-b6d3-f1d74aa430b5 service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:42.731769 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b05b7-9470-41e3-bb23-3bdf5109e15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:42.732685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b05b7-9470-41e3-bb23-3bdf5109e15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:42.732969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b05b7-9470-41e3-bb23-3bdf5109e15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:42.733247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b05b7-9470-41e3-bb23-3bdf5109e15b admin admin] PATH_INFO: `/v3/users/918ac24195b84bd599d66e29e960bcca` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:42.738388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b05b7-9470-41e3-bb23-3bdf5109e15b admin admin] RBAC: Authorizing `identity:delete_user(user_id=918ac24195b84bd599d66e29e960bcca)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:42.741039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b05b7-9470-41e3-bb23-3bdf5109e15b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:42.762937 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:42.764440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:42.769418 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:42.789162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7b4a2-0a5f-4f92-9bbc-86f71bcaf3e6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:42.827955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08d0e7-7ce5-4b61-a18e-c329eb66adeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:42.827955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08d0e7-7ce5-4b61-a18e-c329eb66adeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:42.827955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08d0e7-7ce5-4b61-a18e-c329eb66adeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:42.827955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08d0e7-7ce5-4b61-a18e-c329eb66adeb admin admin] PATH_INFO: `/v3/users/d47c0721137343f2b5600ce5fd7de8d3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:42.846882 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08d0e7-7ce5-4b61-a18e-c329eb66adeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d47c0721137343f2b5600ce5fd7de8d3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:42.852083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-9f08d0e7-7ce5-4b61-a18e-c329eb66adeb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 15:18:42.852083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jun 24 15:18:42.852083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: Jun 24 15:18:42.853349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08d0e7-7ce5-4b61-a18e-c329eb66adeb admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:43.026897 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-974b05b7-9470-41e3-bb23-3bdf5109e15b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '918ac24195b84bd599d66e29e960bcca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:43.068928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-974b05b7-9470-41e3-bb23-3bdf5109e15b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '918ac24195b84bd599d66e29e960bcca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:43.071472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-974b05b7-9470-41e3-bb23-3bdf5109e15b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '918ac24195b84bd599d66e29e960bcca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:43.078444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-974b05b7-9470-41e3-bb23-3bdf5109e15b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '918ac24195b84bd599d66e29e960bcca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:43.079206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-974b05b7-9470-41e3-bb23-3bdf5109e15b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '918ac24195b84bd599d66e29e960bcca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:43.205060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 838/1666] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:18:42 2022] DELETE /identity/v3/users/918ac24195b84bd599d66e29e960bcca => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:43.247312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f08d0e7-7ce5-4b61-a18e-c329eb66adeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd47c0721137343f2b5600ce5fd7de8d3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:43.291013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:43.314819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f08d0e7-7ce5-4b61-a18e-c329eb66adeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd47c0721137343f2b5600ce5fd7de8d3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:43.410884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f08d0e7-7ce5-4b61-a18e-c329eb66adeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd47c0721137343f2b5600ce5fd7de8d3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:43.412631 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f08d0e7-7ce5-4b61-a18e-c329eb66adeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd47c0721137343f2b5600ce5fd7de8d3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:43.428322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f08d0e7-7ce5-4b61-a18e-c329eb66adeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd47c0721137343f2b5600ce5fd7de8d3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:43.436422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 829/1667] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:18:42 2022] DELETE /identity/v3/users/d47c0721137343f2b5600ce5fd7de8d3 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:43.446478 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5e8cb-05a4-4b5d-ab93-f1280093eb5b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:43.448525 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5e8cb-05a4-4b5d-ab93-f1280093eb5b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:43.448525 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5e8cb-05a4-4b5d-ab93-f1280093eb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:43.507023 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1cd5e8cb-05a4-4b5d-ab93-f1280093eb5b None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:43.514072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1cd5e8cb-05a4-4b5d-ab93-f1280093eb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:43.564558 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790331c-68dd-4894-95e9-89e2b45a1e39 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:43.564897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790331c-68dd-4894-95e9-89e2b45a1e39 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:43.565237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790331c-68dd-4894-95e9-89e2b45a1e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:43.678210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1790331c-68dd-4894-95e9-89e2b45a1e39 None None] MFA Rules not processed for user `b67761f9b13340a090efc1cb07714912`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:43.680594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1790331c-68dd-4894-95e9-89e2b45a1e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:43.796166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 839/1668] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:43.817740 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5e8cb-05a4-4b5d-ab93-f1280093eb5b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:43.821762 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1cd5e8cb-05a4-4b5d-ab93-f1280093eb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:43.880104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 830/1669] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:18:43.882643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebed938-b68c-4507-8d94-ec46f8fe7f14 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:43.883102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebed938-b68c-4507-8d94-ec46f8fe7f14 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:43.883554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebed938-b68c-4507-8d94-ec46f8fe7f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:43.967654 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0ebed938-b68c-4507-8d94-ec46f8fe7f14 None None] MFA Rules not processed for user `206ca9f278fc47e4a9a4f720083e91f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:43.968887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0ebed938-b68c-4507-8d94-ec46f8fe7f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:43.975321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5e8cb-05a4-4b5d-ab93-f1280093eb5b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:43.979928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f733a-90b3-4ca8-b99e-ae2f428bfedf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:43.980833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f733a-90b3-4ca8-b99e-ae2f428bfedf service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:43.981264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f733a-90b3-4ca8-b99e-ae2f428bfedf service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:43.981684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f733a-90b3-4ca8-b99e-ae2f428bfedf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:43.997731 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f733a-90b3-4ca8-b99e-ae2f428bfedf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:44.008707 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f733a-90b3-4ca8-b99e-ae2f428bfedf service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:44.042695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 840/1670] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:18:44.044168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2de105-9cd1-4072-919c-5f7b30134b6a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:44.044461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2de105-9cd1-4072-919c-5f7b30134b6a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:44.044813 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2de105-9cd1-4072-919c-5f7b30134b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:44.083368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4e2de105-9cd1-4072-919c-5f7b30134b6a None None] MFA Rules not processed for user `b67761f9b13340a090efc1cb07714912`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:44.084647 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4e2de105-9cd1-4072-919c-5f7b30134b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:44.120155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 841/1671] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:44.219574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 831/1672] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:44.271437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2de105-9cd1-4072-919c-5f7b30134b6a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:44.272919 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4e2de105-9cd1-4072-919c-5f7b30134b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:44.413489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:44.426337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebed938-b68c-4507-8d94-ec46f8fe7f14 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:44.428059 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0ebed938-b68c-4507-8d94-ec46f8fe7f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:44.506076 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2de105-9cd1-4072-919c-5f7b30134b6a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:44.530259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2f50d-4e5c-4487-958b-eb4f69e13571 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:44.531917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2f50d-4e5c-4487-958b-eb4f69e13571 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:44.532851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2f50d-4e5c-4487-958b-eb4f69e13571 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:44.533500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2f50d-4e5c-4487-958b-eb4f69e13571 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:44.544906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2f50d-4e5c-4487-958b-eb4f69e13571 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:44.547988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2f50d-4e5c-4487-958b-eb4f69e13571 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:44.583120 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebed938-b68c-4507-8d94-ec46f8fe7f14 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:44.604688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 842/1673] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:44.605911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2f50d-4e5c-4487-958b-eb4f69e13571 service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:44.618584 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f5e2f50d-4e5c-4487-958b-eb4f69e13571 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:44.622182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3afb9-94ea-4cba-b918-26aec7a023d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:44.628106 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3afb9-94ea-4cba-b918-26aec7a023d5 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:44.628106 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3afb9-94ea-4cba-b918-26aec7a023d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:44.628106 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3afb9-94ea-4cba-b918-26aec7a023d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:44.648634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3afb9-94ea-4cba-b918-26aec7a023d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:44.650401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3afb9-94ea-4cba-b918-26aec7a023d5 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:44.677029 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2f50d-4e5c-4487-958b-eb4f69e13571 service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:44.681051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 832/1674] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:18:44.693083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:44.693083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:44.693083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:44.693906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] PATH_INFO: `/v3/projects/78bb138ba51f4372ad082b535e2160cb` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:44.696931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=78bb138ba51f4372ad082b535e2160cb)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:44.702016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 15:18:44.702016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jun 24 15:18:44.702016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: Jun 24 15:18:44.717199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:44.846461 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:44.963747 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78bb138ba51f4372ad082b535e2160cb', 'request_id': 'req-462d0540-4f69-43bd-a237-8721821d2c6b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:44.998558 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78bb138ba51f4372ad082b535e2160cb', 'request_id': 'req-462d0540-4f69-43bd-a237-8721821d2c6b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:45.096788 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78bb138ba51f4372ad082b535e2160cb', 'request_id': 'req-462d0540-4f69-43bd-a237-8721821d2c6b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:45.124555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] The token cache is being invalidate because project 78bb138ba51f4372ad082b535e2160cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:18:45.126411 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:45.127913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:45.136918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 843/1675] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:18:44 2022] DELETE /identity/v3/projects/78bb138ba51f4372ad082b535e2160cb => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:45.137671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c82190-eb36-49db-b576-b6730042b77f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:45.137961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c82190-eb36-49db-b576-b6730042b77f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:45.138242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c82190-eb36-49db-b576-b6730042b77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:45.188315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c4c82190-eb36-49db-b576-b6730042b77f None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:45.194391 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c4c82190-eb36-49db-b576-b6730042b77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:45.273532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:45.274906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:45.363395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 833/1676] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:45.377794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773857f-1647-43cd-b764-4199d91861a3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:45.378091 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773857f-1647-43cd-b764-4199d91861a3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:45.379840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773857f-1647-43cd-b764-4199d91861a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:45.416577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d0540-4f69-43bd-a237-8721821d2c6b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:45.430818 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff941e-8d21-47fa-8925-6710a822c85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:45.432017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff941e-8d21-47fa-8925-6710a822c85e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:45.432397 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff941e-8d21-47fa-8925-6710a822c85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:45.434366 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3773857f-1647-43cd-b764-4199d91861a3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:45.436119 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3773857f-1647-43cd-b764-4199d91861a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:45.448132 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff941e-8d21-47fa-8925-6710a822c85e admin admin] PATH_INFO: `/v3/projects/34f8d0620ab040d0b20215d0e40ecb97` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:45.449786 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff941e-8d21-47fa-8925-6710a822c85e admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f8d0620ab040d0b20215d0e40ecb97)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:45.455168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff941e-8d21-47fa-8925-6710a822c85e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:45.461679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 834/1677] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:45.463192 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773857f-1647-43cd-b764-4199d91861a3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:45.482671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773857f-1647-43cd-b764-4199d91861a3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:45.487848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c092acf-a635-487c-8d14-8c8495f38f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:45.489899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c092acf-a635-487c-8d14-8c8495f38f26 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:45.490407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c092acf-a635-487c-8d14-8c8495f38f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:45.490841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c092acf-a635-487c-8d14-8c8495f38f26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:45.491892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c092acf-a635-487c-8d14-8c8495f38f26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:18:45.492452 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c092acf-a635-487c-8d14-8c8495f38f26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:45.495720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c092acf-a635-487c-8d14-8c8495f38f26 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:45.525898 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 835/1678] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:18:45.527121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c092acf-a635-487c-8d14-8c8495f38f26 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:45.553860 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c092acf-a635-487c-8d14-8c8495f38f26 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:45.575240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814518d5-bd4e-48d2-b494-7af7ec2ded66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:45.575894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814518d5-bd4e-48d2-b494-7af7ec2ded66 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:45.576339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814518d5-bd4e-48d2-b494-7af7ec2ded66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:45.577018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814518d5-bd4e-48d2-b494-7af7ec2ded66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:45.577868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814518d5-bd4e-48d2-b494-7af7ec2ded66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:45.580328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814518d5-bd4e-48d2-b494-7af7ec2ded66 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:45.667668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f2ff941e-8d21-47fa-8925-6710a822c85e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f8d0620ab040d0b20215d0e40ecb97', 'request_id': 'req-f2ff941e-8d21-47fa-8925-6710a822c85e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:45.686387 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f2ff941e-8d21-47fa-8925-6710a822c85e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f8d0620ab040d0b20215d0e40ecb97', 'request_id': 'req-f2ff941e-8d21-47fa-8925-6710a822c85e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:45.745092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 836/1679] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:45 2022] POST /identity/v3/projects => generated 375 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:18:45.749378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814518d5-bd4e-48d2-b494-7af7ec2ded66 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:45.794495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f2ff941e-8d21-47fa-8925-6710a822c85e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f8d0620ab040d0b20215d0e40ecb97', 'request_id': 'req-f2ff941e-8d21-47fa-8925-6710a822c85e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:45.813466 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f2ff941e-8d21-47fa-8925-6710a822c85e admin admin] The token cache is being invalidate because project 34f8d0620ab040d0b20215d0e40ecb97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:18:45.840283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f2ff941e-8d21-47fa-8925-6710a822c85e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:45.845221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f2ff941e-8d21-47fa-8925-6710a822c85e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:45.866861 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 844/1680] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:18:45 2022] DELETE /identity/v3/projects/34f8d0620ab040d0b20215d0e40ecb97 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:45.871321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff941e-8d21-47fa-8925-6710a822c85e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:45.881647 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f2ff941e-8d21-47fa-8925-6710a822c85e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:45.950812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814518d5-bd4e-48d2-b494-7af7ec2ded66 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:45.973622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d09163-940c-48f9-a768-99a66b691d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:45.973622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d09163-940c-48f9-a768-99a66b691d7d admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:45.973622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d09163-940c-48f9-a768-99a66b691d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:45.973622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d09163-940c-48f9-a768-99a66b691d7d admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:45.974098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d09163-940c-48f9-a768-99a66b691d7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:45.975604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d09163-940c-48f9-a768-99a66b691d7d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:46.073267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 837/1681] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:45 2022] POST /identity/v3/users => generated 288 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:46.078562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22a4b4-6f77-4c91-b57f-b931a6b99163 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:46.082154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22a4b4-6f77-4c91-b57f-b931a6b99163 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:46.084371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22a4b4-6f77-4c91-b57f-b931a6b99163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:46.088446 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff941e-8d21-47fa-8925-6710a822c85e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:46.103741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88c24e-0669-403b-9763-491eddbfe160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:46.105841 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88c24e-0669-403b-9763-491eddbfe160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:46.106582 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88c24e-0669-403b-9763-491eddbfe160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:46.108627 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88c24e-0669-403b-9763-491eddbfe160 admin admin] PATH_INFO: `/v3/users/daa83908246c44e992fe7906de362c71` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:46.112648 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88c24e-0669-403b-9763-491eddbfe160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daa83908246c44e992fe7906de362c71)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:46.122504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88c24e-0669-403b-9763-491eddbfe160 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:46.165623 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7d22a4b4-6f77-4c91-b57f-b931a6b99163 None None] MFA Rules not processed for user `b67761f9b13340a090efc1cb07714912`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:46.167230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7d22a4b4-6f77-4c91-b57f-b931a6b99163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:46.298344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 838/1682] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:46.300823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22a4b4-6f77-4c91-b57f-b931a6b99163 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:46.303833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7d22a4b4-6f77-4c91-b57f-b931a6b99163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:46.340811 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22a4b4-6f77-4c91-b57f-b931a6b99163 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:46.356759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe0b08-6884-4cf0-ae5d-96d8cc175b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:46.358190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe0b08-6884-4cf0-ae5d-96d8cc175b1f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:46.358469 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe0b08-6884-4cf0-ae5d-96d8cc175b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:46.358769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe0b08-6884-4cf0-ae5d-96d8cc175b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:46.359524 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe0b08-6884-4cf0-ae5d-96d8cc175b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:46.366512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe0b08-6884-4cf0-ae5d-96d8cc175b1f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:46.396551 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 839/1683] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:46.401182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe0b08-6884-4cf0-ae5d-96d8cc175b1f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:46.415816 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe0b08-6884-4cf0-ae5d-96d8cc175b1f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:46.422470 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252635d-4a09-4888-b161-1fb9969957a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:46.423454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252635d-4a09-4888-b161-1fb9969957a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:46.424942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252635d-4a09-4888-b161-1fb9969957a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:46.425335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252635d-4a09-4888-b161-1fb9969957a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:46.428767 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252635d-4a09-4888-b161-1fb9969957a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:46.437045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252635d-4a09-4888-b161-1fb9969957a0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:46.459695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:46.464401 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1e88c24e-0669-403b-9763-491eddbfe160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daa83908246c44e992fe7906de362c71'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:46.479302 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 840/1684] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:46.481072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252635d-4a09-4888-b161-1fb9969957a0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:46.496722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1e88c24e-0669-403b-9763-491eddbfe160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daa83908246c44e992fe7906de362c71'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:46.498342 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252635d-4a09-4888-b161-1fb9969957a0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:46.499787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1e88c24e-0669-403b-9763-491eddbfe160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daa83908246c44e992fe7906de362c71'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:46.514702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec9fa8-aa18-4abc-b8e8-5564abe17e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:46.517040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec9fa8-aa18-4abc-b8e8-5564abe17e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:46.517543 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec9fa8-aa18-4abc-b8e8-5564abe17e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:46.519740 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec9fa8-aa18-4abc-b8e8-5564abe17e2e admin admin] PATH_INFO: `/v3/projects/db09a8c1aa9c45848310796420c9233c/users/47a883838ce74582be4272143fdee208/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:46.533819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1e88c24e-0669-403b-9763-491eddbfe160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daa83908246c44e992fe7906de362c71'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:46.535603 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1e88c24e-0669-403b-9763-491eddbfe160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daa83908246c44e992fe7906de362c71'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:46.545032 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec9fa8-aa18-4abc-b8e8-5564abe17e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=db09a8c1aa9c45848310796420c9233c, user_id=47a883838ce74582be4272143fdee208, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:46.550269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec9fa8-aa18-4abc-b8e8-5564abe17e2e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:46.675365 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 845/1685] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:18:45 2022] DELETE /identity/v3/users/daa83908246c44e992fe7906de362c71 => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:46.687709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d2cb7-4119-4c3c-88b9-93f170716e19 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:46.688005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d2cb7-4119-4c3c-88b9-93f170716e19 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:46.688285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d2cb7-4119-4c3c-88b9-93f170716e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:46.693151 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 841/1686] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:46 2022] PUT /identity/v3/projects/db09a8c1aa9c45848310796420c9233c/users/47a883838ce74582be4272143fdee208/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:46.762275 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-848d2cb7-4119-4c3c-88b9-93f170716e19 None None] MFA Rules not processed for user `47a883838ce74582be4272143fdee208`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:46.764892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-848d2cb7-4119-4c3c-88b9-93f170716e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:46.834044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:46.906958 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 846/1687] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:46.907752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec9fa8-aa18-4abc-b8e8-5564abe17e2e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:46.910017 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a8ec9fa8-aa18-4abc-b8e8-5564abe17e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:47.125411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec9fa8-aa18-4abc-b8e8-5564abe17e2e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:47.274933 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec343b64-7725-49d0-822f-5eb20aa4d8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:47.275986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec343b64-7725-49d0-822f-5eb20aa4d8ef admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:47.279237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec343b64-7725-49d0-822f-5eb20aa4d8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:47.279589 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec343b64-7725-49d0-822f-5eb20aa4d8ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:47.280887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec343b64-7725-49d0-822f-5eb20aa4d8ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:47.286056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec343b64-7725-49d0-822f-5eb20aa4d8ef admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:47.469198 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 842/1688] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:46 2022] POST /identity/v3/projects => generated 377 bytes in 563 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:47.473927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d2cb7-4119-4c3c-88b9-93f170716e19 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:47.515525 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d2cb7-4119-4c3c-88b9-93f170716e19 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:47.636837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e1052-4866-4a66-8be5-4c36b523b4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:47.637762 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e1052-4866-4a66-8be5-4c36b523b4a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:47.638241 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e1052-4866-4a66-8be5-4c36b523b4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:47.638578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e1052-4866-4a66-8be5-4c36b523b4a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:47.639462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e1052-4866-4a66-8be5-4c36b523b4a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:47.642144 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e1052-4866-4a66-8be5-4c36b523b4a5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:47.761073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 847/1689] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:47 2022] POST /identity/v3/users => generated 295 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:47.764973 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec343b64-7725-49d0-822f-5eb20aa4d8ef admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:47.780238 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec343b64-7725-49d0-822f-5eb20aa4d8ef admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:47.786720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee45f9-f9c2-4d4f-bdd7-3b18c02f97c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:47.788083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee45f9-f9c2-4d4f-bdd7-3b18c02f97c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:47.788569 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee45f9-f9c2-4d4f-bdd7-3b18c02f97c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:47.789070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee45f9-f9c2-4d4f-bdd7-3b18c02f97c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:47.790700 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee45f9-f9c2-4d4f-bdd7-3b18c02f97c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:47.792664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee45f9-f9c2-4d4f-bdd7-3b18c02f97c6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:47.808100 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 843/1690] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:47.812066 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e1052-4866-4a66-8be5-4c36b523b4a5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:47.829880 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e1052-4866-4a66-8be5-4c36b523b4a5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:47.839475 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d166c-1c62-4868-804d-1a05435a75fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:47.841297 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d166c-1c62-4868-804d-1a05435a75fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:47.842150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d166c-1c62-4868-804d-1a05435a75fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:47.842587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d166c-1c62-4868-804d-1a05435a75fe admin admin] PATH_INFO: `/v3/projects/d4750ba043044c8280e8e0239df72a4e/users/43581f76c4d540108494a84337dcb152/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:47.880672 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d166c-1c62-4868-804d-1a05435a75fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4750ba043044c8280e8e0239df72a4e, user_id=43581f76c4d540108494a84337dcb152, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:47.891489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d166c-1c62-4868-804d-1a05435a75fe admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:48.008861 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 848/1691] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:47 2022] PUT /identity/v3/projects/d4750ba043044c8280e8e0239df72a4e/users/43581f76c4d540108494a84337dcb152/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:48.017920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee45f9-f9c2-4d4f-bdd7-3b18c02f97c6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:48.050532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee45f9-f9c2-4d4f-bdd7-3b18c02f97c6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:48.093315 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d166c-1c62-4868-804d-1a05435a75fe admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:48.095739 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2f6d166c-1c62-4868-804d-1a05435a75fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:48.288959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f6db5-e718-420b-ab33-314bdd708dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:48.300842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f6db5-e718-420b-ab33-314bdd708dff admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:48.301810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f6db5-e718-420b-ab33-314bdd708dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:48.302181 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f6db5-e718-420b-ab33-314bdd708dff admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:48.303075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f6db5-e718-420b-ab33-314bdd708dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:48.304783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f6db5-e718-420b-ab33-314bdd708dff admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:48.397121 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d166c-1c62-4868-804d-1a05435a75fe admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:48.398571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 844/1692] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:48.399660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f6db5-e718-420b-ab33-314bdd708dff admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:48.407105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9486162-5504-4400-82a7-4ee81b23e3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:48.410986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9486162-5504-4400-82a7-4ee81b23e3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:48.412291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9486162-5504-4400-82a7-4ee81b23e3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:48.412793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9486162-5504-4400-82a7-4ee81b23e3ec admin admin] PATH_INFO: `/v3/projects/39ce535da8614cc8a423316e7077eace` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:48.416588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9486162-5504-4400-82a7-4ee81b23e3ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ce535da8614cc8a423316e7077eace)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:48.426022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9486162-5504-4400-82a7-4ee81b23e3ec admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:48.434202 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f6db5-e718-420b-ab33-314bdd708dff admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:48.479647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2125fe-5410-47ad-96c9-1a35ef5dd200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:48.481429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2125fe-5410-47ad-96c9-1a35ef5dd200 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:48.481878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2125fe-5410-47ad-96c9-1a35ef5dd200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:48.483533 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2125fe-5410-47ad-96c9-1a35ef5dd200 admin admin] PATH_INFO: `/v3/projects/d4750ba043044c8280e8e0239df72a4e/users/43581f76c4d540108494a84337dcb152/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:48.485124 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2125fe-5410-47ad-96c9-1a35ef5dd200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4750ba043044c8280e8e0239df72a4e, user_id=43581f76c4d540108494a84337dcb152, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:48.506923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2125fe-5410-47ad-96c9-1a35ef5dd200 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:48.542032 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 845/1693] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:48 2022] PUT /identity/v3/projects/d4750ba043044c8280e8e0239df72a4e/users/43581f76c4d540108494a84337dcb152/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:48.548634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a562b6-c3a3-4632-80d4-23001cb424b1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:48.551897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a562b6-c3a3-4632-80d4-23001cb424b1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:48.553620 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a562b6-c3a3-4632-80d4-23001cb424b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:48.648625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e9486162-5504-4400-82a7-4ee81b23e3ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ce535da8614cc8a423316e7077eace', 'request_id': 'req-e9486162-5504-4400-82a7-4ee81b23e3ec'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:48.678535 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e9486162-5504-4400-82a7-4ee81b23e3ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ce535da8614cc8a423316e7077eace', 'request_id': 'req-e9486162-5504-4400-82a7-4ee81b23e3ec'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:48.689319 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-49a562b6-c3a3-4632-80d4-23001cb424b1 None None] MFA Rules not processed for user `43581f76c4d540108494a84337dcb152`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:48.695568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-49a562b6-c3a3-4632-80d4-23001cb424b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:48.777120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e9486162-5504-4400-82a7-4ee81b23e3ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ce535da8614cc8a423316e7077eace', 'request_id': 'req-e9486162-5504-4400-82a7-4ee81b23e3ec'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:48.786562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:48.798920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e9486162-5504-4400-82a7-4ee81b23e3ec admin admin] The token cache is being invalidate because project 39ce535da8614cc8a423316e7077eace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:18:48.800807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e9486162-5504-4400-82a7-4ee81b23e3ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:48.809821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e9486162-5504-4400-82a7-4ee81b23e3ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:48.813489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 849/1694] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:18:48 2022] DELETE /identity/v3/projects/39ce535da8614cc8a423316e7077eace => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:48.820711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00551726-4dd0-4c4b-b254-e30a530d21bf None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:48.821215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00551726-4dd0-4c4b-b254-e30a530d21bf None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:48.821647 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00551726-4dd0-4c4b-b254-e30a530d21bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:48.896756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-00551726-4dd0-4c4b-b254-e30a530d21bf None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:48.900320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-00551726-4dd0-4c4b-b254-e30a530d21bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:49.006862 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 846/1695] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:49.008512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eae8ed-052a-4af1-8bc7-3e1c2b52352d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:49.008994 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eae8ed-052a-4af1-8bc7-3e1c2b52352d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:49.009458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eae8ed-052a-4af1-8bc7-3e1c2b52352d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:49.029935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d3eae8ed-052a-4af1-8bc7-3e1c2b52352d None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:49.032006 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d3eae8ed-052a-4af1-8bc7-3e1c2b52352d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:49.042630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:49.118450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 847/1696] 10.222.0.67 () {58 vars in 1009 bytes} [Fri Jun 24 15:18:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:49.119703 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eae8ed-052a-4af1-8bc7-3e1c2b52352d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:49.121348 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d3eae8ed-052a-4af1-8bc7-3e1c2b52352d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:49.157429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 850/1697] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:49.159424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0d169-87a7-4b25-822e-fe3ed74c28b3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:49.163701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0d169-87a7-4b25-822e-fe3ed74c28b3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:49.174909 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0d169-87a7-4b25-822e-fe3ed74c28b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:49.193552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ddc0d169-87a7-4b25-822e-fe3ed74c28b3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:49.195864 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ddc0d169-87a7-4b25-822e-fe3ed74c28b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:49.207789 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 851/1698] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:49.218773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0d169-87a7-4b25-822e-fe3ed74c28b3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:49.233679 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0d169-87a7-4b25-822e-fe3ed74c28b3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:49.242386 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8986a5-7de7-4eb5-899b-f42e1195e052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:49.245713 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8986a5-7de7-4eb5-899b-f42e1195e052 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:49.245713 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8986a5-7de7-4eb5-899b-f42e1195e052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:49.245713 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8986a5-7de7-4eb5-899b-f42e1195e052 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:49.246930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8986a5-7de7-4eb5-899b-f42e1195e052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:18:49.247712 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8986a5-7de7-4eb5-899b-f42e1195e052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:49.250733 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8986a5-7de7-4eb5-899b-f42e1195e052 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:49.263124 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eae8ed-052a-4af1-8bc7-3e1c2b52352d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:49.280492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84334d2-ca98-472e-8489-01f359799229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:49.281741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84334d2-ca98-472e-8489-01f359799229 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:49.282337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84334d2-ca98-472e-8489-01f359799229 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:49.282689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84334d2-ca98-472e-8489-01f359799229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:49.287057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a84334d2-ca98-472e-8489-01f359799229 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:49.295678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 852/1699] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:18:49.296966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8986a5-7de7-4eb5-899b-f42e1195e052 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:49.299281 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7a8986a5-7de7-4eb5-899b-f42e1195e052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:49.320034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8986a5-7de7-4eb5-899b-f42e1195e052 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:49.334704 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4df934-909a-4f07-9738-c2b9c960fe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:49.336319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4df934-909a-4f07-9738-c2b9c960fe00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:49.340844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4df934-909a-4f07-9738-c2b9c960fe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:49.340844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4df934-909a-4f07-9738-c2b9c960fe00 admin admin] PATH_INFO: `/v3/users/a3cd1f48d64f47868ed582f79a7906f7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:49.343760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4df934-909a-4f07-9738-c2b9c960fe00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3cd1f48d64f47868ed582f79a7906f7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:49.347023 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4df934-909a-4f07-9738-c2b9c960fe00 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:49.443874 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84334d2-ca98-472e-8489-01f359799229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:49.445605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84334d2-ca98-472e-8489-01f359799229 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:49.470473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 848/1700] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:49.474058 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84334d2-ca98-472e-8489-01f359799229 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:49.493066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84334d2-ca98-472e-8489-01f359799229 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:49.501976 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3064299-73d0-418f-a676-94e83e7f7a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:49.503069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3064299-73d0-418f-a676-94e83e7f7a68 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:49.503559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3064299-73d0-418f-a676-94e83e7f7a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:49.505003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3064299-73d0-418f-a676-94e83e7f7a68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:49.506756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3064299-73d0-418f-a676-94e83e7f7a68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:49.513196 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3064299-73d0-418f-a676-94e83e7f7a68 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:49.623126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 849/1701] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:49 2022] POST /identity/v3/projects => generated 409 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:49.624050 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3064299-73d0-418f-a676-94e83e7f7a68 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:49.628699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c3064299-73d0-418f-a676-94e83e7f7a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:49.656854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-df4df934-909a-4f07-9738-c2b9c960fe00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3cd1f48d64f47868ed582f79a7906f7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:49.690688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-df4df934-909a-4f07-9738-c2b9c960fe00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3cd1f48d64f47868ed582f79a7906f7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:49.693286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-df4df934-909a-4f07-9738-c2b9c960fe00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3cd1f48d64f47868ed582f79a7906f7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:49.702962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-df4df934-909a-4f07-9738-c2b9c960fe00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3cd1f48d64f47868ed582f79a7906f7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:49.703941 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-df4df934-909a-4f07-9738-c2b9c960fe00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3cd1f48d64f47868ed582f79a7906f7'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:49.820169 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 853/1702] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:18:49 2022] DELETE /identity/v3/users/a3cd1f48d64f47868ed582f79a7906f7 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:49.825448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4df934-909a-4f07-9738-c2b9c960fe00 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:49.827657 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-df4df934-909a-4f07-9738-c2b9c960fe00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:49.889201 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3064299-73d0-418f-a676-94e83e7f7a68 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:50.027886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4df934-909a-4f07-9738-c2b9c960fe00 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:50.039425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326f8e2-9853-4dfc-8f23-275397194f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:50.043590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326f8e2-9853-4dfc-8f23-275397194f84 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:50.043590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326f8e2-9853-4dfc-8f23-275397194f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:50.043590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326f8e2-9853-4dfc-8f23-275397194f84 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:50.045136 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326f8e2-9853-4dfc-8f23-275397194f84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:50.048344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326f8e2-9853-4dfc-8f23-275397194f84 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:50.065288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f96277-d807-477f-94d1-915a7d785fab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:50.066235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f96277-d807-477f-94d1-915a7d785fab service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:50.067835 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f96277-d807-477f-94d1-915a7d785fab service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:50.068688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f96277-d807-477f-94d1-915a7d785fab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:50.071643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-44f96277-d807-477f-94d1-915a7d785fab service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:50.122541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:50.160332 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 854/1703] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:49 2022] POST /identity/v3/users => generated 305 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:50.163548 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efc432-72e5-4425-970f-c2c03fe1a765 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:50.164541 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efc432-72e5-4425-970f-c2c03fe1a765 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:50.171786 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efc432-72e5-4425-970f-c2c03fe1a765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:50.205918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-95efc432-72e5-4425-970f-c2c03fe1a765 None None] MFA Rules not processed for user `206ca9f278fc47e4a9a4f720083e91f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:50.208217 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-95efc432-72e5-4425-970f-c2c03fe1a765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:50.221713 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f96277-d807-477f-94d1-915a7d785fab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:50.223419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f96277-d807-477f-94d1-915a7d785fab service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:50.244790 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 850/1704] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:18:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:18:50.246262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f57436-7e29-4f86-adb6-27ed831b9781 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:50.249637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f57436-7e29-4f86-adb6-27ed831b9781 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:50.249637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f57436-7e29-4f86-adb6-27ed831b9781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:50.305164 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-27f57436-7e29-4f86-adb6-27ed831b9781 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:50.311663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-27f57436-7e29-4f86-adb6-27ed831b9781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:50.348853 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 851/1705] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:50.348853 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f57436-7e29-4f86-adb6-27ed831b9781 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:50.364320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 855/1706] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:50.388985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f57436-7e29-4f86-adb6-27ed831b9781 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:50.421700 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662e19a-bcd7-42dd-a351-663560ade8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:50.422398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662e19a-bcd7-42dd-a351-663560ade8fc admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:50.422684 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662e19a-bcd7-42dd-a351-663560ade8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:50.422966 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662e19a-bcd7-42dd-a351-663560ade8fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:50.423737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662e19a-bcd7-42dd-a351-663560ade8fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:50.425370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662e19a-bcd7-42dd-a351-663560ade8fc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:50.529626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 852/1707] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:50.531709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efc432-72e5-4425-970f-c2c03fe1a765 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:50.576491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efc432-72e5-4425-970f-c2c03fe1a765 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:50.611536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcac79-3bae-40d6-9e26-9bf91d0fba90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:50.612986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcac79-3bae-40d6-9e26-9bf91d0fba90 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:50.614758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcac79-3bae-40d6-9e26-9bf91d0fba90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:50.616425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcac79-3bae-40d6-9e26-9bf91d0fba90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:50.618783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcac79-3bae-40d6-9e26-9bf91d0fba90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:50.622602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcac79-3bae-40d6-9e26-9bf91d0fba90 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:50.650447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662e19a-bcd7-42dd-a351-663560ade8fc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:50.652756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e662e19a-bcd7-42dd-a351-663560ade8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:50.668605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 856/1708] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:18:50.678927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcac79-3bae-40d6-9e26-9bf91d0fba90 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:50.712902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:50.735502 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcac79-3bae-40d6-9e26-9bf91d0fba90 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:50.765570 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3713d-3640-475f-8395-8d373e0bc306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:50.766954 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3713d-3640-475f-8395-8d373e0bc306 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:50.771886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3713d-3640-475f-8395-8d373e0bc306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:50.773199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3713d-3640-475f-8395-8d373e0bc306 admin admin] PATH_INFO: `/v3/projects/1be816b1d9264d72804cb89fac91ae48/users/9d2a85c1f30c4ecb9d342495c08d3d17/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:50.849151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3713d-3640-475f-8395-8d373e0bc306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1be816b1d9264d72804cb89fac91ae48, user_id=9d2a85c1f30c4ecb9d342495c08d3d17, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:50.854389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3713d-3640-475f-8395-8d373e0bc306 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:50.860737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662e19a-bcd7-42dd-a351-663560ade8fc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:50.874360 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7a129-fe76-4aba-bf2b-75f6c76059a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:50.877054 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7a129-fe76-4aba-bf2b-75f6c76059a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:50.882477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7a129-fe76-4aba-bf2b-75f6c76059a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:50.884026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7a129-fe76-4aba-bf2b-75f6c76059a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:50.910008 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:50.920833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7a129-fe76-4aba-bf2b-75f6c76059a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:50.922765 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7a129-fe76-4aba-bf2b-75f6c76059a7 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:50.970990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 857/1709] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:50 2022] PUT /identity/v3/projects/1be816b1d9264d72804cb89fac91ae48/users/9d2a85c1f30c4ecb9d342495c08d3d17/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:50.972941 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcf6d1-6572-4a60-a2db-adfb6a0e8e4c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:50.973536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcf6d1-6572-4a60-a2db-adfb6a0e8e4c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:50.974818 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcf6d1-6572-4a60-a2db-adfb6a0e8e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:50.999680 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 853/1710] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:51.001035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7a129-fe76-4aba-bf2b-75f6c76059a7 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:51.009632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-86f7a129-fe76-4aba-bf2b-75f6c76059a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:51.068386 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-69dcf6d1-6572-4a60-a2db-adfb6a0e8e4c None None] MFA Rules not processed for user `9d2a85c1f30c4ecb9d342495c08d3d17`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:51.073820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-69dcf6d1-6572-4a60-a2db-adfb6a0e8e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:51.083579 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7a129-fe76-4aba-bf2b-75f6c76059a7 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:51.284658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 858/1711] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:50 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:51.286045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcf6d1-6572-4a60-a2db-adfb6a0e8e4c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:51.323654 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06a68d-2373-4eab-819c-75a3b416b295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:51.325704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06a68d-2373-4eab-819c-75a3b416b295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:51.326738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06a68d-2373-4eab-819c-75a3b416b295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:51.331439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06a68d-2373-4eab-819c-75a3b416b295 admin admin] PATH_INFO: `/v3/users/32f1810ce2ae432685c4d7f642e43eb2` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:51.338654 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06a68d-2373-4eab-819c-75a3b416b295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32f1810ce2ae432685c4d7f642e43eb2)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:51.348123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06a68d-2373-4eab-819c-75a3b416b295 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:51.632816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcf6d1-6572-4a60-a2db-adfb6a0e8e4c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:51.657847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0486c-7b0e-41d9-b98a-af7855ca63d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:51.657847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0486c-7b0e-41d9-b98a-af7855ca63d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:51.663441 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0486c-7b0e-41d9-b98a-af7855ca63d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:51.665080 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0486c-7b0e-41d9-b98a-af7855ca63d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:51.667768 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0486c-7b0e-41d9-b98a-af7855ca63d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:51.669791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0486c-7b0e-41d9-b98a-af7855ca63d8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:51.761430 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:51.812845 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fa06a68d-2373-4eab-819c-75a3b416b295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32f1810ce2ae432685c4d7f642e43eb2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:51.856565 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 859/1712] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:51 2022] POST /identity/v3/projects => generated 407 bytes in 571 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:51.860353 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0486c-7b0e-41d9-b98a-af7855ca63d8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:51.890972 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fa06a68d-2373-4eab-819c-75a3b416b295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32f1810ce2ae432685c4d7f642e43eb2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:52.097066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fa06a68d-2373-4eab-819c-75a3b416b295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32f1810ce2ae432685c4d7f642e43eb2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:52.098809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fa06a68d-2373-4eab-819c-75a3b416b295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32f1810ce2ae432685c4d7f642e43eb2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:52.114167 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fa06a68d-2373-4eab-819c-75a3b416b295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32f1810ce2ae432685c4d7f642e43eb2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:52.133912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0486c-7b0e-41d9-b98a-af7855ca63d8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:52.143434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 854/1713] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:18:50 2022] DELETE /identity/v3/users/32f1810ce2ae432685c4d7f642e43eb2 => generated 0 bytes in 1143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:52.150744 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c4894-c07f-4ce5-b16c-33f83800613d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:52.158830 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c4894-c07f-4ce5-b16c-33f83800613d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:52.158830 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c4894-c07f-4ce5-b16c-33f83800613d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:52.228959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-028c4894-c07f-4ce5-b16c-33f83800613d None None] MFA Rules not processed for user `95e7fb8f48dd4348b54e3694a8410bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:52.240046 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-028c4894-c07f-4ce5-b16c-33f83800613d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:52.248794 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:52.370386 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81cb5e-7db5-4e17-b619-3896be34cbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:52.371660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81cb5e-7db5-4e17-b619-3896be34cbfe admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:52.372033 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81cb5e-7db5-4e17-b619-3896be34cbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:52.372592 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81cb5e-7db5-4e17-b619-3896be34cbfe admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:52.373412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81cb5e-7db5-4e17-b619-3896be34cbfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:52.382328 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81cb5e-7db5-4e17-b619-3896be34cbfe admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:52.475346 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 855/1714] 10.222.0.64 () {58 vars in 1015 bytes} [Fri Jun 24 15:18:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:52.475450 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 860/1715] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:51 2022] POST /identity/v3/users => generated 310 bytes in 610 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:52.475541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f51c1-0961-474f-bfd2-fd4c44aa007f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:52.475541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f51c1-0961-474f-bfd2-fd4c44aa007f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:52.475541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f51c1-0961-474f-bfd2-fd4c44aa007f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:52.475749 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81cb5e-7db5-4e17-b619-3896be34cbfe admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:52.482245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8e81cb5e-7db5-4e17-b619-3896be34cbfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:52.525419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81cb5e-7db5-4e17-b619-3896be34cbfe admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:52.532316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-694f51c1-0961-474f-bfd2-fd4c44aa007f None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:52.537341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-694f51c1-0961-474f-bfd2-fd4c44aa007f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:52.543211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380c564-203a-4a4b-94eb-0ed67ae10652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:52.544093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380c564-203a-4a4b-94eb-0ed67ae10652 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:52.544477 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380c564-203a-4a4b-94eb-0ed67ae10652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:52.544852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380c564-203a-4a4b-94eb-0ed67ae10652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:52.545688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380c564-203a-4a4b-94eb-0ed67ae10652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:52.552360 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380c564-203a-4a4b-94eb-0ed67ae10652 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:52.566400 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 856/1716] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:52.572484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f51c1-0961-474f-bfd2-fd4c44aa007f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:52.572484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-694f51c1-0961-474f-bfd2-fd4c44aa007f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:52.585884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 861/1717] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:18:52.588372 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380c564-203a-4a4b-94eb-0ed67ae10652 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:52.612940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f51c1-0961-474f-bfd2-fd4c44aa007f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:52.614888 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380c564-203a-4a4b-94eb-0ed67ae10652 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:52.622892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88dbfc-9693-4adc-8e65-c032b6377a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:52.623686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88dbfc-9693-4adc-8e65-c032b6377a3e service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:52.623979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88dbfc-9693-4adc-8e65-c032b6377a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:52.624260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88dbfc-9693-4adc-8e65-c032b6377a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:52.633091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e24483-8126-437d-abed-48c7190e8023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:52.633778 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e24483-8126-437d-abed-48c7190e8023 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:52.634060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e24483-8126-437d-abed-48c7190e8023 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:52.634353 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e24483-8126-437d-abed-48c7190e8023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:52.662489 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88dbfc-9693-4adc-8e65-c032b6377a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:52.663990 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88dbfc-9693-4adc-8e65-c032b6377a3e service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:52.695015 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e24483-8126-437d-abed-48c7190e8023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:52.714399 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e24483-8126-437d-abed-48c7190e8023 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:52.728357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 857/1718] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:52.733096 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88dbfc-9693-4adc-8e65-c032b6377a3e service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:52.763534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 862/1719] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:52.773892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88dbfc-9693-4adc-8e65-c032b6377a3e service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:52.797347 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7112563-c1f3-4078-9689-5576e80fe6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:52.800241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7112563-c1f3-4078-9689-5576e80fe6af admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:52.801690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7112563-c1f3-4078-9689-5576e80fe6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:52.802426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7112563-c1f3-4078-9689-5576e80fe6af admin admin] PATH_INFO: `/v3/projects/6a28bb1df9134de9a179b6b8021b8d5d/users/27902c80add140c1b83fe26316b6f73e/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:52.849036 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7112563-c1f3-4078-9689-5576e80fe6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a28bb1df9134de9a179b6b8021b8d5d, user_id=27902c80add140c1b83fe26316b6f73e, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:52.857955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7112563-c1f3-4078-9689-5576e80fe6af admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:52.961548 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 858/1720] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:52 2022] PUT /identity/v3/projects/6a28bb1df9134de9a179b6b8021b8d5d/users/27902c80add140c1b83fe26316b6f73e/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:52.965629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e24483-8126-437d-abed-48c7190e8023 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:52.990354 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7112563-c1f3-4078-9689-5576e80fe6af admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:52.990854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b7112563-c1f3-4078-9689-5576e80fe6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:52.992699 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e24483-8126-437d-abed-48c7190e8023 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:53.137260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:53.150131 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7112563-c1f3-4078-9689-5576e80fe6af admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:53.152756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac3506-4116-4033-ba7b-ac7843624f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:53.153461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac3506-4116-4033-ba7b-ac7843624f4a admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:53.153753 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac3506-4116-4033-ba7b-ac7843624f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:53.154038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac3506-4116-4033-ba7b-ac7843624f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:53.154786 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac3506-4116-4033-ba7b-ac7843624f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:53.160251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d00cf-80f6-4a58-860d-666914e22668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:53.161823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac3506-4116-4033-ba7b-ac7843624f4a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:53.163730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d00cf-80f6-4a58-860d-666914e22668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:53.163730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d00cf-80f6-4a58-860d-666914e22668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:53.163730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d00cf-80f6-4a58-860d-666914e22668 admin admin] PATH_INFO: `/v3/users/860003a2d0e04181b812a5e72b3ffca9` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:53.164897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d00cf-80f6-4a58-860d-666914e22668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=860003a2d0e04181b812a5e72b3ffca9)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:53.167061 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d00cf-80f6-4a58-860d-666914e22668 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:53.176603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:53.196994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 863/1721] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:53.201601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac3506-4116-4033-ba7b-ac7843624f4a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:53.213890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac3506-4116-4033-ba7b-ac7843624f4a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:53.221728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576572b-adf7-4079-8415-2716e912f7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:53.223153 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576572b-adf7-4079-8415-2716e912f7a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:53.223715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576572b-adf7-4079-8415-2716e912f7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:53.224132 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576572b-adf7-4079-8415-2716e912f7a8 admin admin] PATH_INFO: `/v3/projects/6a28bb1df9134de9a179b6b8021b8d5d/users/27902c80add140c1b83fe26316b6f73e/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:53.226818 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576572b-adf7-4079-8415-2716e912f7a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a28bb1df9134de9a179b6b8021b8d5d, user_id=27902c80add140c1b83fe26316b6f73e, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:53.230573 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576572b-adf7-4079-8415-2716e912f7a8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:53.253082 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 864/1722] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:53 2022] PUT /identity/v3/projects/6a28bb1df9134de9a179b6b8021b8d5d/users/27902c80add140c1b83fe26316b6f73e/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:18:53.269670 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc78e40-5cce-4584-a8ce-b507dc8db171 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:53.271555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc78e40-5cce-4584-a8ce-b507dc8db171 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:53.272321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc78e40-5cce-4584-a8ce-b507dc8db171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:53.356472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-bbc78e40-5cce-4584-a8ce-b507dc8db171 None None] MFA Rules not processed for user `27902c80add140c1b83fe26316b6f73e`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:53.357849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bbc78e40-5cce-4584-a8ce-b507dc8db171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:53.418548 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-145d00cf-80f6-4a58-860d-666914e22668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '860003a2d0e04181b812a5e72b3ffca9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:53.448634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-145d00cf-80f6-4a58-860d-666914e22668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '860003a2d0e04181b812a5e72b3ffca9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:53.464761 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:53.552214 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-145d00cf-80f6-4a58-860d-666914e22668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '860003a2d0e04181b812a5e72b3ffca9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:53.554793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-145d00cf-80f6-4a58-860d-666914e22668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '860003a2d0e04181b812a5e72b3ffca9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:53.558881 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 865/1723] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:53 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:53.561616 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc78e40-5cce-4584-a8ce-b507dc8db171 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:53.565605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-145d00cf-80f6-4a58-860d-666914e22668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '860003a2d0e04181b812a5e72b3ffca9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:53.567656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bbc78e40-5cce-4584-a8ce-b507dc8db171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:53.588784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 859/1724] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:18:52 2022] DELETE /identity/v3/users/860003a2d0e04181b812a5e72b3ffca9 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:53.593245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d00cf-80f6-4a58-860d-666914e22668 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:53.595449 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-145d00cf-80f6-4a58-860d-666914e22668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:53.727766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc78e40-5cce-4584-a8ce-b507dc8db171 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:53.759358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d00cf-80f6-4a58-860d-666914e22668 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:53.760074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:53.794924 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82da13b-898b-4299-b737-315a781d1426 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:53.795611 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82da13b-898b-4299-b737-315a781d1426 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:53.796056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82da13b-898b-4299-b737-315a781d1426 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:53.796487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82da13b-898b-4299-b737-315a781d1426 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:53.800694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a82da13b-898b-4299-b737-315a781d1426 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:53.993774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82da13b-898b-4299-b737-315a781d1426 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:53.998817 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82da13b-898b-4299-b737-315a781d1426 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:54.006079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ccbe5-5849-4d98-ab4b-d50830d3388d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:54.016188 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ccbe5-5849-4d98-ab4b-d50830d3388d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:54.035590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ccbe5-5849-4d98-ab4b-d50830d3388d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:54.038908 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ccbe5-5849-4d98-ab4b-d50830d3388d admin admin] PATH_INFO: `/v3/projects/3326c29b85f643c28253c9ce948bda8a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:54.044313 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ccbe5-5849-4d98-ab4b-d50830d3388d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3326c29b85f643c28253c9ce948bda8a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:54.059493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ccbe5-5849-4d98-ab4b-d50830d3388d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:54.077785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 860/1725] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:53 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:54.089101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c2684-c9b0-41b7-a9a9-d2a8b169f3fa None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:54.090270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c2684-c9b0-41b7-a9a9-d2a8b169f3fa None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:54.090270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c2684-c9b0-41b7-a9a9-d2a8b169f3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:54.196798 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f57c2684-c9b0-41b7-a9a9-d2a8b169f3fa None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:54.198055 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f57c2684-c9b0-41b7-a9a9-d2a8b169f3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:54.215829 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 861/1726] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:54.240247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c2684-c9b0-41b7-a9a9-d2a8b169f3fa None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:54.246531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f57c2684-c9b0-41b7-a9a9-d2a8b169f3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:54.269531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fe9ccbe5-5849-4d98-ab4b-d50830d3388d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3326c29b85f643c28253c9ce948bda8a', 'request_id': 'req-fe9ccbe5-5849-4d98-ab4b-d50830d3388d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:54.298227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fe9ccbe5-5849-4d98-ab4b-d50830d3388d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3326c29b85f643c28253c9ce948bda8a', 'request_id': 'req-fe9ccbe5-5849-4d98-ab4b-d50830d3388d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:54.379328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:54.382248 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fe9ccbe5-5849-4d98-ab4b-d50830d3388d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3326c29b85f643c28253c9ce948bda8a', 'request_id': 'req-fe9ccbe5-5849-4d98-ab4b-d50830d3388d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:54.409729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fe9ccbe5-5849-4d98-ab4b-d50830d3388d admin admin] The token cache is being invalidate because project 3326c29b85f643c28253c9ce948bda8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:18:54.410859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fe9ccbe5-5849-4d98-ab4b-d50830d3388d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:54.411645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fe9ccbe5-5849-4d98-ab4b-d50830d3388d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:54.417906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 866/1727] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:18:53 2022] DELETE /identity/v3/projects/3326c29b85f643c28253c9ce948bda8a => generated 0 bytes in 855 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:54.422503 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc0a13-17f7-4f9e-9ed0-6ede4b3a225a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:54.423286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc0a13-17f7-4f9e-9ed0-6ede4b3a225a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:54.423943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc0a13-17f7-4f9e-9ed0-6ede4b3a225a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:54.454765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d5fc0a13-17f7-4f9e-9ed0-6ede4b3a225a None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:54.456166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d5fc0a13-17f7-4f9e-9ed0-6ede4b3a225a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:54.456493 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c2684-c9b0-41b7-a9a9-d2a8b169f3fa None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:54.602575 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31386837-4850-4cb6-92ef-16984f070096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:54.603393 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31386837-4850-4cb6-92ef-16984f070096 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:54.603681 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31386837-4850-4cb6-92ef-16984f070096 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:54.603973 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31386837-4850-4cb6-92ef-16984f070096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:54.610445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-31386837-4850-4cb6-92ef-16984f070096 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:54.634773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 867/1728] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:18:54.649080 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46303b-5be4-4d2e-aa4f-7dcbcb1471dd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:54.653365 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46303b-5be4-4d2e-aa4f-7dcbcb1471dd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:54.653365 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46303b-5be4-4d2e-aa4f-7dcbcb1471dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:54.712852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1a46303b-5be4-4d2e-aa4f-7dcbcb1471dd None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:54.721246 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1a46303b-5be4-4d2e-aa4f-7dcbcb1471dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:54.731364 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 868/1729] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:54.757199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46303b-5be4-4d2e-aa4f-7dcbcb1471dd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:54.798300 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46303b-5be4-4d2e-aa4f-7dcbcb1471dd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:54.812927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7b5da-e977-4e8b-856a-6b02da2e80db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:54.812927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7b5da-e977-4e8b-856a-6b02da2e80db admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:54.812927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7b5da-e977-4e8b-856a-6b02da2e80db admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:54.830148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7b5da-e977-4e8b-856a-6b02da2e80db admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:54.832515 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31386837-4850-4cb6-92ef-16984f070096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:54.834224 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31386837-4850-4cb6-92ef-16984f070096 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:54.834628 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7b5da-e977-4e8b-856a-6b02da2e80db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:18:54.836207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7b5da-e977-4e8b-856a-6b02da2e80db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:54.848034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7b5da-e977-4e8b-856a-6b02da2e80db admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:54.851786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 862/1730] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:54.853958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31386837-4850-4cb6-92ef-16984f070096 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:54.859099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-31386837-4850-4cb6-92ef-16984f070096 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:54.878237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 869/1731] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:18:54.891296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7b5da-e977-4e8b-856a-6b02da2e80db admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:54.917501 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7b5da-e977-4e8b-856a-6b02da2e80db admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:54.929197 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c9f63-31dd-4cb7-855e-76517bb7afb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:54.930039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c9f63-31dd-4cb7-855e-76517bb7afb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:54.933602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c9f63-31dd-4cb7-855e-76517bb7afb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:54.933602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c9f63-31dd-4cb7-855e-76517bb7afb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:54.933602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c9f63-31dd-4cb7-855e-76517bb7afb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:54.939343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c9f63-31dd-4cb7-855e-76517bb7afb4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:54.978869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31386837-4850-4cb6-92ef-16984f070096 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:54.987489 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d749e-5459-4a4d-9334-2052847d6aa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:54.990278 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d749e-5459-4a4d-9334-2052847d6aa2 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:54.990815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d749e-5459-4a4d-9334-2052847d6aa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:54.991251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d749e-5459-4a4d-9334-2052847d6aa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:54.995474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-852d749e-5459-4a4d-9334-2052847d6aa2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:55.135799 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:55.153026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 870/1732] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:54 2022] POST /identity/v3/projects => generated 365 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:55.155285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c9f63-31dd-4cb7-855e-76517bb7afb4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:55.189067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:55.204014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d749e-5459-4a4d-9334-2052847d6aa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:55.221513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d749e-5459-4a4d-9334-2052847d6aa2 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:55.261827 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 863/1733] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:55.401681 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c9f63-31dd-4cb7-855e-76517bb7afb4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:55.432099 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9299e-9d3d-41ab-898e-e364465b6cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:55.432816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9299e-9d3d-41ab-898e-e364465b6cef admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:55.433101 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9299e-9d3d-41ab-898e-e364465b6cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:55.433379 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9299e-9d3d-41ab-898e-e364465b6cef admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:55.434913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9299e-9d3d-41ab-898e-e364465b6cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:55.438926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9299e-9d3d-41ab-898e-e364465b6cef admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:55.568087 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 871/1734] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:55 2022] POST /identity/v3/users => generated 283 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:55.572380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d749e-5459-4a4d-9334-2052847d6aa2 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:55.590201 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d749e-5459-4a4d-9334-2052847d6aa2 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:55.603316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443328a-af57-4513-b354-c89317bc9583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:55.604226 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443328a-af57-4513-b354-c89317bc9583 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:55.605940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443328a-af57-4513-b354-c89317bc9583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:55.619780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443328a-af57-4513-b354-c89317bc9583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:55.624952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443328a-af57-4513-b354-c89317bc9583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:55.632179 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443328a-af57-4513-b354-c89317bc9583 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:55.678287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 864/1735] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:55.679652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9299e-9d3d-41ab-898e-e364465b6cef admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:55.694627 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9299e-9d3d-41ab-898e-e364465b6cef admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:55.712663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c7885-abc9-4978-8862-d158c0355303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:55.717979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c7885-abc9-4978-8862-d158c0355303 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:55.718432 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c7885-abc9-4978-8862-d158c0355303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:55.719093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c7885-abc9-4978-8862-d158c0355303 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:55.720743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c7885-abc9-4978-8862-d158c0355303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:55.729603 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c7885-abc9-4978-8862-d158c0355303 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:55.761587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 872/1736] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:55.763694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443328a-af57-4513-b354-c89317bc9583 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:55.787248 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443328a-af57-4513-b354-c89317bc9583 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:55.793865 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ddb3c-60f3-40bb-8f32-e71d8fe75c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:55.794875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ddb3c-60f3-40bb-8f32-e71d8fe75c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:55.795294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ddb3c-60f3-40bb-8f32-e71d8fe75c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:55.795646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ddb3c-60f3-40bb-8f32-e71d8fe75c1b admin admin] PATH_INFO: `/v3/projects/35a0890819894fd49a8476c5b013cb19/users/43ab5fc92e2c4dadaa137e6f2aa42a69/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:55.822621 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c7885-abc9-4978-8862-d158c0355303 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:55.824649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-720c7885-abc9-4978-8862-d158c0355303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:55.845942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ddb3c-60f3-40bb-8f32-e71d8fe75c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=35a0890819894fd49a8476c5b013cb19, user_id=43ab5fc92e2c4dadaa137e6f2aa42a69, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:55.871987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ddb3c-60f3-40bb-8f32-e71d8fe75c1b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:55.876694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c7885-abc9-4978-8862-d158c0355303 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:55.893792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8de9f6-bdf3-4ee9-9bfa-075522199484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:55.895734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8de9f6-bdf3-4ee9-9bfa-075522199484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:55.896468 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8de9f6-bdf3-4ee9-9bfa-075522199484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:55.896848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8de9f6-bdf3-4ee9-9bfa-075522199484 admin admin] PATH_INFO: `/v3/projects/6ea548217ba14fa4aabbfe81fa916275` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:55.899032 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8de9f6-bdf3-4ee9-9bfa-075522199484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ea548217ba14fa4aabbfe81fa916275)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:55.908892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8de9f6-bdf3-4ee9-9bfa-075522199484 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:56.006520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 865/1737] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:55 2022] PUT /identity/v3/projects/35a0890819894fd49a8476c5b013cb19/users/43ab5fc92e2c4dadaa137e6f2aa42a69/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:56.016963 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c517e-bc08-4b36-941d-4395cdac48b1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:56.017466 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c517e-bc08-4b36-941d-4395cdac48b1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:56.017758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c517e-bc08-4b36-941d-4395cdac48b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:56.050447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ca6c517e-bc08-4b36-941d-4395cdac48b1 None None] MFA Rules not processed for user `43ab5fc92e2c4dadaa137e6f2aa42a69`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:56.052324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fd8de9f6-bdf3-4ee9-9bfa-075522199484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ea548217ba14fa4aabbfe81fa916275', 'request_id': 'req-fd8de9f6-bdf3-4ee9-9bfa-075522199484'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:56.055332 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ca6c517e-bc08-4b36-941d-4395cdac48b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:56.061444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fd8de9f6-bdf3-4ee9-9bfa-075522199484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ea548217ba14fa4aabbfe81fa916275', 'request_id': 'req-fd8de9f6-bdf3-4ee9-9bfa-075522199484'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:56.182449 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 866/1738] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:18:56.184425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fd8de9f6-bdf3-4ee9-9bfa-075522199484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ea548217ba14fa4aabbfe81fa916275', 'request_id': 'req-fd8de9f6-bdf3-4ee9-9bfa-075522199484'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:56.185119 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c517e-bc08-4b36-941d-4395cdac48b1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:56.208183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fd8de9f6-bdf3-4ee9-9bfa-075522199484 admin admin] The token cache is being invalidate because project 6ea548217ba14fa4aabbfe81fa916275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:18:56.208569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fd8de9f6-bdf3-4ee9-9bfa-075522199484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:56.209367 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fd8de9f6-bdf3-4ee9-9bfa-075522199484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:56.225584 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 873/1739] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:18:55 2022] DELETE /identity/v3/projects/6ea548217ba14fa4aabbfe81fa916275 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:18:56.241020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8de9f6-bdf3-4ee9-9bfa-075522199484 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:56.256962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fd8de9f6-bdf3-4ee9-9bfa-075522199484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:56.313709 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c517e-bc08-4b36-941d-4395cdac48b1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:56.320865 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d399e7-c8fd-4a20-8971-8b2e22a3ec0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:56.321813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d399e7-c8fd-4a20-8971-8b2e22a3ec0b admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:56.322433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d399e7-c8fd-4a20-8971-8b2e22a3ec0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:56.322909 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d399e7-c8fd-4a20-8971-8b2e22a3ec0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:56.324016 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d399e7-c8fd-4a20-8971-8b2e22a3ec0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:56.326247 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d399e7-c8fd-4a20-8971-8b2e22a3ec0b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:56.536930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8de9f6-bdf3-4ee9-9bfa-075522199484 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:56.551334 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 867/1740] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:56 2022] POST /identity/v3/projects => generated 365 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:18:56.552572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237919a4-73d4-4ff7-9ac1-d012f32b0770 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:56.552869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237919a4-73d4-4ff7-9ac1-d012f32b0770 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:56.553218 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237919a4-73d4-4ff7-9ac1-d012f32b0770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:56.574761 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13d404-0800-4f7a-82e4-777e6891c4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:56.575702 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13d404-0800-4f7a-82e4-777e6891c4d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:56.575986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13d404-0800-4f7a-82e4-777e6891c4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:56.583159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13d404-0800-4f7a-82e4-777e6891c4d0 admin admin] PATH_INFO: `/v3/projects/ebc1917e3470485696bfd6e97d67b5e5` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:56.586328 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13d404-0800-4f7a-82e4-777e6891c4d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebc1917e3470485696bfd6e97d67b5e5)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:56.588327 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13d404-0800-4f7a-82e4-777e6891c4d0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:56.620230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:56.655673 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-237919a4-73d4-4ff7-9ac1-d012f32b0770 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:56.657186 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-237919a4-73d4-4ff7-9ac1-d012f32b0770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:56.718774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:56.803913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ea13d404-0800-4f7a-82e4-777e6891c4d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc1917e3470485696bfd6e97d67b5e5', 'request_id': 'req-ea13d404-0800-4f7a-82e4-777e6891c4d0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:56.837394 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ea13d404-0800-4f7a-82e4-777e6891c4d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc1917e3470485696bfd6e97d67b5e5', 'request_id': 'req-ea13d404-0800-4f7a-82e4-777e6891c4d0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:56.848763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 868/1741] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:56.848763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237919a4-73d4-4ff7-9ac1-d012f32b0770 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:56.848763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-237919a4-73d4-4ff7-9ac1-d012f32b0770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:56.924925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237919a4-73d4-4ff7-9ac1-d012f32b0770 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:56.929785 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ea13d404-0800-4f7a-82e4-777e6891c4d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc1917e3470485696bfd6e97d67b5e5', 'request_id': 'req-ea13d404-0800-4f7a-82e4-777e6891c4d0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:56.955905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ea13d404-0800-4f7a-82e4-777e6891c4d0 admin admin] The token cache is being invalidate because project ebc1917e3470485696bfd6e97d67b5e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:18:56.961294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ea13d404-0800-4f7a-82e4-777e6891c4d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:56.964725 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ea13d404-0800-4f7a-82e4-777e6891c4d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:56.974043 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 874/1742] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:18:56 2022] DELETE /identity/v3/projects/ebc1917e3470485696bfd6e97d67b5e5 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:56.974043 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5162d3-be56-4c85-8c91-14ebe610d573 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:56.974043 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5162d3-be56-4c85-8c91-14ebe610d573 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:56.974043 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5162d3-be56-4c85-8c91-14ebe610d573 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:56.988816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 875/1743] 10.222.0.67 () {54 vars in 921 bytes} [Fri Jun 24 15:18:56 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:18:56.989693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5162d3-be56-4c85-8c91-14ebe610d573 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:56.999650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5a5162d3-be56-4c85-8c91-14ebe610d573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:57.016395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479b0d0-0f52-41c5-b6ab-6d9af2854942 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:57.018279 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479b0d0-0f52-41c5-b6ab-6d9af2854942 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:57.018740 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479b0d0-0f52-41c5-b6ab-6d9af2854942 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:57.019700 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479b0d0-0f52-41c5-b6ab-6d9af2854942 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:57.023208 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5479b0d0-0f52-41c5-b6ab-6d9af2854942 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:57.182140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479b0d0-0f52-41c5-b6ab-6d9af2854942 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:57.183949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479b0d0-0f52-41c5-b6ab-6d9af2854942 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:57.209094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 869/1744] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:57.210456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671cc00-0bf5-4b23-a9c5-8289ac330204 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:57.210740 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671cc00-0bf5-4b23-a9c5-8289ac330204 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:57.211016 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671cc00-0bf5-4b23-a9c5-8289ac330204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:57.245538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5162d3-be56-4c85-8c91-14ebe610d573 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:57.247313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b671cc00-0bf5-4b23-a9c5-8289ac330204 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:57.254957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b671cc00-0bf5-4b23-a9c5-8289ac330204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:57.271697 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36305f20-fb23-4c4b-8445-43e585cfcf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:57.275888 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36305f20-fb23-4c4b-8445-43e585cfcf39 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:57.275888 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36305f20-fb23-4c4b-8445-43e585cfcf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:57.276587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36305f20-fb23-4c4b-8445-43e585cfcf39 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:57.277882 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36305f20-fb23-4c4b-8445-43e585cfcf39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:57.278649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 870/1745] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:57.279552 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671cc00-0bf5-4b23-a9c5-8289ac330204 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:57.282779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b671cc00-0bf5-4b23-a9c5-8289ac330204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:57.287578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36305f20-fb23-4c4b-8445-43e585cfcf39 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:57.306922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671cc00-0bf5-4b23-a9c5-8289ac330204 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:57.316087 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddb460-8836-4dba-8b52-76f8448996f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:57.317544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddb460-8836-4dba-8b52-76f8448996f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:57.317867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddb460-8836-4dba-8b52-76f8448996f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:57.323523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddb460-8836-4dba-8b52-76f8448996f6 admin admin] PATH_INFO: `/v3/users/7946e39bcfa54daea7459a3173965317` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:57.326313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ddb460-8836-4dba-8b52-76f8448996f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7946e39bcfa54daea7459a3173965317)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:57.334607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ddb460-8836-4dba-8b52-76f8448996f6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:57.385441 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:57.482684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 876/1746] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:56 2022] POST /identity/v3/users => generated 289 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:57.482684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43069dd-6fe4-480a-b8c4-147f822115fc None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:57.482684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43069dd-6fe4-480a-b8c4-147f822115fc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:57.482684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43069dd-6fe4-480a-b8c4-147f822115fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:57.525310 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d43069dd-6fe4-480a-b8c4-147f822115fc None None] MFA Rules not processed for user `51de59caa3144a1cbf12b20b15e2134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:57.527338 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d43069dd-6fe4-480a-b8c4-147f822115fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:57.607169 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-98ddb460-8836-4dba-8b52-76f8448996f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7946e39bcfa54daea7459a3173965317'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:57.641102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-98ddb460-8836-4dba-8b52-76f8448996f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7946e39bcfa54daea7459a3173965317'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:57.670462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 877/1747] 10.222.0.67 () {58 vars in 1018 bytes} [Fri Jun 24 15:18:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:57.671981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43069dd-6fe4-480a-b8c4-147f822115fc None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:57.684334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43069dd-6fe4-480a-b8c4-147f822115fc None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:57.697353 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b9671-e7da-4c05-b737-e826845e4d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:57.698039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b9671-e7da-4c05-b737-e826845e4d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:57.698371 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b9671-e7da-4c05-b737-e826845e4d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:57.698656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b9671-e7da-4c05-b737-e826845e4d58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:57.699419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b9671-e7da-4c05-b737-e826845e4d58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:18:57.699728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b9671-e7da-4c05-b737-e826845e4d58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:57.705693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b9671-e7da-4c05-b737-e826845e4d58 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:57.729833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-98ddb460-8836-4dba-8b52-76f8448996f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7946e39bcfa54daea7459a3173965317'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:57.730468 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-98ddb460-8836-4dba-8b52-76f8448996f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7946e39bcfa54daea7459a3173965317'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:57.740089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-98ddb460-8836-4dba-8b52-76f8448996f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7946e39bcfa54daea7459a3173965317'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:57.747746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 878/1748] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:18:57.748903 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b9671-e7da-4c05-b737-e826845e4d58 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:57.754463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 871/1749] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:18:57 2022] DELETE /identity/v3/users/7946e39bcfa54daea7459a3173965317 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:57.756060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-025b9671-e7da-4c05-b737-e826845e4d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:57.756525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddb460-8836-4dba-8b52-76f8448996f6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:57.761070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-98ddb460-8836-4dba-8b52-76f8448996f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:57.778255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:57.785490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b9671-e7da-4c05-b737-e826845e4d58 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:57.896919 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d66873-edcd-4fb9-80dd-2ab1a2d15669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:57.897728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d66873-edcd-4fb9-80dd-2ab1a2d15669 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:57.898116 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d66873-edcd-4fb9-80dd-2ab1a2d15669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:57.898624 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d66873-edcd-4fb9-80dd-2ab1a2d15669 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:57.899765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d66873-edcd-4fb9-80dd-2ab1a2d15669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:57.903597 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d66873-edcd-4fb9-80dd-2ab1a2d15669 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:57.925085 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 879/1750] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:57.925888 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d66873-edcd-4fb9-80dd-2ab1a2d15669 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:57.935639 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d2d66873-edcd-4fb9-80dd-2ab1a2d15669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:57.956299 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddb460-8836-4dba-8b52-76f8448996f6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:57.967988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13148a5-e65e-43c7-894c-9032302ab515 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:57.968710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d66873-edcd-4fb9-80dd-2ab1a2d15669 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:57.979636 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13148a5-e65e-43c7-894c-9032302ab515 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:57.979982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13148a5-e65e-43c7-894c-9032302ab515 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:57.980270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13148a5-e65e-43c7-894c-9032302ab515 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:57.986251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f13148a5-e65e-43c7-894c-9032302ab515 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:58.002197 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07834cd4-0d36-4d26-90a3-c983d711c2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:58.004155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07834cd4-0d36-4d26-90a3-c983d711c2f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:58.004997 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07834cd4-0d36-4d26-90a3-c983d711c2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:58.005606 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07834cd4-0d36-4d26-90a3-c983d711c2f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:58.007361 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07834cd4-0d36-4d26-90a3-c983d711c2f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:58.009895 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07834cd4-0d36-4d26-90a3-c983d711c2f0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:58.038551 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13148a5-e65e-43c7-894c-9032302ab515 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:58.040764 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13148a5-e65e-43c7-894c-9032302ab515 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:58.106944 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 872/1751] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:58.107949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13148a5-e65e-43c7-894c-9032302ab515 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:58.109069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 880/1752] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:57 2022] POST /identity/v3/projects => generated 405 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:18:58.114299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07834cd4-0d36-4d26-90a3-c983d711c2f0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:58.134480 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13148a5-e65e-43c7-894c-9032302ab515 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:58.344625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07834cd4-0d36-4d26-90a3-c983d711c2f0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:58.359573 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbfb06e-c992-4691-8bbd-9d2817d853d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:58.362469 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfb06e-c992-4691-8bbd-9d2817d853d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:58.362746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfb06e-c992-4691-8bbd-9d2817d853d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:58.363635 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfb06e-c992-4691-8bbd-9d2817d853d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:58.368571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbfb06e-c992-4691-8bbd-9d2817d853d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:58.379549 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbfb06e-c992-4691-8bbd-9d2817d853d8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:58.413260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7793d-eb7e-4185-b91f-5dbd24492a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:58.418122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7793d-eb7e-4185-b91f-5dbd24492a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:58.419135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7793d-eb7e-4185-b91f-5dbd24492a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:58.421159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7793d-eb7e-4185-b91f-5dbd24492a42 admin admin] PATH_INFO: `/v3/projects/5a9e9230178c457cb0a3d4691c6bf2e9/users/4b33f52acd374efbaf7ff7730cc32f3e/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:58.517876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7793d-eb7e-4185-b91f-5dbd24492a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a9e9230178c457cb0a3d4691c6bf2e9, user_id=4b33f52acd374efbaf7ff7730cc32f3e, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:58.536003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7793d-eb7e-4185-b91f-5dbd24492a42 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:58.561802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 881/1753] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:58 2022] POST /identity/v3/users => generated 303 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:18:58.566097 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbfb06e-c992-4691-8bbd-9d2817d853d8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:58.604039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbfb06e-c992-4691-8bbd-9d2817d853d8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:58.610638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588a706-a395-4c2f-95dd-b63bf259ac21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:58.611760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588a706-a395-4c2f-95dd-b63bf259ac21 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:58.612142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588a706-a395-4c2f-95dd-b63bf259ac21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:58.612516 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588a706-a395-4c2f-95dd-b63bf259ac21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:58.613714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588a706-a395-4c2f-95dd-b63bf259ac21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:58.615665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588a706-a395-4c2f-95dd-b63bf259ac21 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:58.653944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 882/1754] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:18:58.656455 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 873/1755] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:58 2022] PUT /identity/v3/projects/5a9e9230178c457cb0a3d4691c6bf2e9/users/4b33f52acd374efbaf7ff7730cc32f3e/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:58.660946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588a706-a395-4c2f-95dd-b63bf259ac21 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:58.661264 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7793d-eb7e-4185-b91f-5dbd24492a42 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:58.678230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:58.691056 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588a706-a395-4c2f-95dd-b63bf259ac21 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:58.778373 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7793d-eb7e-4185-b91f-5dbd24492a42 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:58.792458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23b13e-5861-4cef-8654-941ed31221ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:58.793152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23b13e-5861-4cef-8654-941ed31221ee admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:58.793451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23b13e-5861-4cef-8654-941ed31221ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:58.793723 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23b13e-5861-4cef-8654-941ed31221ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:58.794754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23b13e-5861-4cef-8654-941ed31221ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:58.796083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23b13e-5861-4cef-8654-941ed31221ee admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:58.812655 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 874/1756] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:18:58.814434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23b13e-5861-4cef-8654-941ed31221ee admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:58.826362 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23b13e-5861-4cef-8654-941ed31221ee admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:58.831451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416bb81-0ca4-4d31-8b2e-fc7a2d555a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:58.833002 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416bb81-0ca4-4d31-8b2e-fc7a2d555a6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:58.833377 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416bb81-0ca4-4d31-8b2e-fc7a2d555a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:58.833661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416bb81-0ca4-4d31-8b2e-fc7a2d555a6a admin admin] PATH_INFO: `/v3/projects/5a9e9230178c457cb0a3d4691c6bf2e9/users/4b33f52acd374efbaf7ff7730cc32f3e/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:58.836258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416bb81-0ca4-4d31-8b2e-fc7a2d555a6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a9e9230178c457cb0a3d4691c6bf2e9, user_id=4b33f52acd374efbaf7ff7730cc32f3e, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:58.849195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416bb81-0ca4-4d31-8b2e-fc7a2d555a6a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:58.849550 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:58.854528 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 875/1757] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:58 2022] PUT /identity/v3/projects/5a9e9230178c457cb0a3d4691c6bf2e9/users/4b33f52acd374efbaf7ff7730cc32f3e/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:58.875647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c769bcc-2bed-4f9f-ad1b-ad13df9e4a4f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:58.876166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba7ae6-0778-40ef-b92f-f197d008b0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:58.877752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c769bcc-2bed-4f9f-ad1b-ad13df9e4a4f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:58.877752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c769bcc-2bed-4f9f-ad1b-ad13df9e4a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:58.878513 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba7ae6-0778-40ef-b92f-f197d008b0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:58.879542 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba7ae6-0778-40ef-b92f-f197d008b0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:58.880442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba7ae6-0778-40ef-b92f-f197d008b0a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:58.881653 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba7ae6-0778-40ef-b92f-f197d008b0a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:58.886503 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba7ae6-0778-40ef-b92f-f197d008b0a8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:58.908351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 883/1758] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:18:58.916474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0c769bcc-2bed-4f9f-ad1b-ad13df9e4a4f None None] MFA Rules not processed for user `4b33f52acd374efbaf7ff7730cc32f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:58.917930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0c769bcc-2bed-4f9f-ad1b-ad13df9e4a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:58.920337 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba7ae6-0778-40ef-b92f-f197d008b0a8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:58.948979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba7ae6-0778-40ef-b92f-f197d008b0a8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:59.036425 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 876/1759] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:58 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:59.037316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c769bcc-2bed-4f9f-ad1b-ad13df9e4a4f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:59.042583 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0c769bcc-2bed-4f9f-ad1b-ad13df9e4a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:59.069874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbae1a0-c861-4292-84a2-1f2d8f099d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:59.069874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbae1a0-c861-4292-84a2-1f2d8f099d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:59.069874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbae1a0-c861-4292-84a2-1f2d8f099d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:59.069874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbae1a0-c861-4292-84a2-1f2d8f099d06 admin admin] PATH_INFO: `/v3/projects/3ec0d7bcce024fbf94e21ccfcb936d7e/users/4ebbf52381904a2b8e8ef8f67d7f3d3a/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:59.100329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbae1a0-c861-4292-84a2-1f2d8f099d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec0d7bcce024fbf94e21ccfcb936d7e, user_id=4ebbf52381904a2b8e8ef8f67d7f3d3a, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:59.103138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbae1a0-c861-4292-84a2-1f2d8f099d06 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:59.176659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c769bcc-2bed-4f9f-ad1b-ad13df9e4a4f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:59.182930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473831f-7b8f-47b7-a611-29e31fa46b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:59.183926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473831f-7b8f-47b7-a611-29e31fa46b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:59.184211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473831f-7b8f-47b7-a611-29e31fa46b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:59.184488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473831f-7b8f-47b7-a611-29e31fa46b4e admin admin] PATH_INFO: `/v3/projects/a207aa7fd3764fe1b8741958d32c8bc6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:59.186508 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473831f-7b8f-47b7-a611-29e31fa46b4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a207aa7fd3764fe1b8741958d32c8bc6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:59.190996 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473831f-7b8f-47b7-a611-29e31fa46b4e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:59.196199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 884/1760] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:18:58 2022] PUT /identity/v3/projects/3ec0d7bcce024fbf94e21ccfcb936d7e/users/4ebbf52381904a2b8e8ef8f67d7f3d3a/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:18:59.197696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbae1a0-c861-4292-84a2-1f2d8f099d06 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:59.199873 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4bbae1a0-c861-4292-84a2-1f2d8f099d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:59.283080 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbae1a0-c861-4292-84a2-1f2d8f099d06 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:59.288587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057f1c7-f604-41dc-a58b-5516c697cbc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:59.290268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057f1c7-f604-41dc-a58b-5516c697cbc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:59.290552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057f1c7-f604-41dc-a58b-5516c697cbc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:59.290837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057f1c7-f604-41dc-a58b-5516c697cbc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:59.369614 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d473831f-7b8f-47b7-a611-29e31fa46b4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a207aa7fd3764fe1b8741958d32c8bc6', 'request_id': 'req-d473831f-7b8f-47b7-a611-29e31fa46b4e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:59.380018 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057f1c7-f604-41dc-a58b-5516c697cbc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:59.381731 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057f1c7-f604-41dc-a58b-5516c697cbc8 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:59.384375 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d473831f-7b8f-47b7-a611-29e31fa46b4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a207aa7fd3764fe1b8741958d32c8bc6', 'request_id': 'req-d473831f-7b8f-47b7-a611-29e31fa46b4e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:59.396145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 885/1761] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:59 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:59.397905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711fa4e-103f-4682-a7b7-6a43a851b152 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:59.397905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711fa4e-103f-4682-a7b7-6a43a851b152 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:59.398201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711fa4e-103f-4682-a7b7-6a43a851b152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:59.421935 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d711fa4e-103f-4682-a7b7-6a43a851b152 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:59.423213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d711fa4e-103f-4682-a7b7-6a43a851b152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:59.462446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d473831f-7b8f-47b7-a611-29e31fa46b4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a207aa7fd3764fe1b8741958d32c8bc6', 'request_id': 'req-d473831f-7b8f-47b7-a611-29e31fa46b4e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:59.480327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d473831f-7b8f-47b7-a611-29e31fa46b4e admin admin] The token cache is being invalidate because project a207aa7fd3764fe1b8741958d32c8bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:18:59.480700 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d473831f-7b8f-47b7-a611-29e31fa46b4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:59.481350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d473831f-7b8f-47b7-a611-29e31fa46b4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:18:59.486947 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 877/1762] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:18:59 2022] DELETE /identity/v3/projects/a207aa7fd3764fe1b8741958d32c8bc6 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:18:59.488913 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5ccae-d8e3-4ce6-be55-e64d012e7adf None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:59.489202 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5ccae-d8e3-4ce6-be55-e64d012e7adf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:59.489477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5ccae-d8e3-4ce6-be55-e64d012e7adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:59.504372 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:59.515916 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 886/1763] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:59.520014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-51c5ccae-d8e3-4ce6-be55-e64d012e7adf None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:59.521326 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-51c5ccae-d8e3-4ce6-be55-e64d012e7adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:59.524288 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711fa4e-103f-4682-a7b7-6a43a851b152 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:59.525759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d711fa4e-103f-4682-a7b7-6a43a851b152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:59.540277 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 878/1764] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:18:59.546072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb8bf9-02b4-473e-ad56-5c1d25d48395 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:59.546072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb8bf9-02b4-473e-ad56-5c1d25d48395 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:59.546072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb8bf9-02b4-473e-ad56-5c1d25d48395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:59.574261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2fbb8bf9-02b4-473e-ad56-5c1d25d48395 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:59.574261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2fbb8bf9-02b4-473e-ad56-5c1d25d48395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:59.589932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 879/1765] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:59.591025 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb8bf9-02b4-473e-ad56-5c1d25d48395 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:59.598838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:59.605411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb8bf9-02b4-473e-ad56-5c1d25d48395 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:59.611555 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419bc26-2acf-43af-acc5-1ef7f33a473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:59.612427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419bc26-2acf-43af-acc5-1ef7f33a473d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:59.613630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419bc26-2acf-43af-acc5-1ef7f33a473d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:59.613961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419bc26-2acf-43af-acc5-1ef7f33a473d admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:59.614839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419bc26-2acf-43af-acc5-1ef7f33a473d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:18:59.615157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419bc26-2acf-43af-acc5-1ef7f33a473d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:59.615746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711fa4e-103f-4682-a7b7-6a43a851b152 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:59.617715 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419bc26-2acf-43af-acc5-1ef7f33a473d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:59.620588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd16ca-2b91-434a-9ddb-b345159d69e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:59.621411 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd16ca-2b91-434a-9ddb-b345159d69e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:59.621816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd16ca-2b91-434a-9ddb-b345159d69e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:59.622187 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd16ca-2b91-434a-9ddb-b345159d69e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:59.640398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 880/1766] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:18:59.645814 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419bc26-2acf-43af-acc5-1ef7f33a473d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:59.648420 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd16ca-2b91-434a-9ddb-b345159d69e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:59.650618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd16ca-2b91-434a-9ddb-b345159d69e7 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:59.663321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 887/1767] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:59.664286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd16ca-2b91-434a-9ddb-b345159d69e7 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:59.665223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419bc26-2acf-43af-acc5-1ef7f33a473d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:59.665463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-30cd16ca-2b91-434a-9ddb-b345159d69e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:59.671777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74a0f3-da1f-4c6b-889a-a8eb53969c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:59.672523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74a0f3-da1f-4c6b-889a-a8eb53969c10 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:59.672874 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74a0f3-da1f-4c6b-889a-a8eb53969c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:59.673263 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74a0f3-da1f-4c6b-889a-a8eb53969c10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:59.674033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74a0f3-da1f-4c6b-889a-a8eb53969c10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:59.676184 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74a0f3-da1f-4c6b-889a-a8eb53969c10 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:59.744510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd16ca-2b91-434a-9ddb-b345159d69e7 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:59.749347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2db7a-5185-4641-bb14-ec08076a568a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:59.750252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2db7a-5185-4641-bb14-ec08076a568a service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:59.750692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2db7a-5185-4641-bb14-ec08076a568a service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:59.751261 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2db7a-5185-4641-bb14-ec08076a568a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:59.753538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-64b2db7a-5185-4641-bb14-ec08076a568a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:59.828089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 881/1768] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:18:59 2022] POST /identity/v3/projects => generated 369 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:18:59.833163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0a704-c4dd-43b5-9293-60b5ca2d9e3c None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:59.833163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0a704-c4dd-43b5-9293-60b5ca2d9e3c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:59.833163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0a704-c4dd-43b5-9293-60b5ca2d9e3c None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:59.833431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2db7a-5185-4641-bb14-ec08076a568a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:59.835045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2db7a-5185-4641-bb14-ec08076a568a service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:59.837570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 882/1769] 10.222.0.67 () {54 vars in 921 bytes} [Fri Jun 24 15:18:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:18:59.838445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0a704-c4dd-43b5-9293-60b5ca2d9e3c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:59.846591 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 888/1770] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:18:59.848630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfad032-43e3-4a68-8999-c695ff02d131 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:59.849171 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfad032-43e3-4a68-8999-c695ff02d131 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:59.849541 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfad032-43e3-4a68-8999-c695ff02d131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:59.869784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9dfad032-43e3-4a68-8999-c695ff02d131 None None] MFA Rules not processed for user `51de59caa3144a1cbf12b20b15e2134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:18:59.871281 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9dfad032-43e3-4a68-8999-c695ff02d131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:59.939468 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0a704-c4dd-43b5-9293-60b5ca2d9e3c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:18:59.945272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3db1bc-ba2a-4772-855f-0e26afa4cee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:18:59.945958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3db1bc-ba2a-4772-855f-0e26afa4cee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:18:59.946244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3db1bc-ba2a-4772-855f-0e26afa4cee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:18:59.946522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3db1bc-ba2a-4772-855f-0e26afa4cee3 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:18:59.947295 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3db1bc-ba2a-4772-855f-0e26afa4cee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:18:59.949170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3db1bc-ba2a-4772-855f-0e26afa4cee3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:18:59.955218 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 889/1771] 10.222.0.67 () {58 vars in 1018 bytes} [Fri Jun 24 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:18:59.971296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfad032-43e3-4a68-8999-c695ff02d131 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:18:59.975561 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9dfad032-43e3-4a68-8999-c695ff02d131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:18:59.995796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfad032-43e3-4a68-8999-c695ff02d131 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:00.001353 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa4605-c293-4e93-9628-27f68b995024 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:00.002014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa4605-c293-4e93-9628-27f68b995024 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:00.002419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa4605-c293-4e93-9628-27f68b995024 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:00.002777 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa4605-c293-4e93-9628-27f68b995024 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:00.011818 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa4605-c293-4e93-9628-27f68b995024 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:00.013425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa4605-c293-4e93-9628-27f68b995024 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:00.024176 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 890/1772] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:18:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:00.077056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 883/1773] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:18:59 2022] POST /identity/v3/users => generated 285 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:00.078289 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa4605-c293-4e93-9628-27f68b995024 service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:00.090022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa4605-c293-4e93-9628-27f68b995024 service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:00.102664 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc74bc-f538-4c12-bf6d-db6486ab6f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:00.103454 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc74bc-f538-4c12-bf6d-db6486ab6f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:00.106678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc74bc-f538-4c12-bf6d-db6486ab6f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:00.106678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc74bc-f538-4c12-bf6d-db6486ab6f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:00.106678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc74bc-f538-4c12-bf6d-db6486ab6f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:00.106678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc74bc-f538-4c12-bf6d-db6486ab6f57 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:00.118031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 891/1774] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:19:00.121559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3db1bc-ba2a-4772-855f-0e26afa4cee3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:00.132038 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3db1bc-ba2a-4772-855f-0e26afa4cee3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:00.136836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86686a2-3697-41f8-9b5b-648336a9af95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:00.137510 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86686a2-3697-41f8-9b5b-648336a9af95 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:00.137826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86686a2-3697-41f8-9b5b-648336a9af95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:00.138105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86686a2-3697-41f8-9b5b-648336a9af95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:00.138824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86686a2-3697-41f8-9b5b-648336a9af95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:00.140537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86686a2-3697-41f8-9b5b-648336a9af95 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:00.151964 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 884/1775] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:00.158282 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc74bc-f538-4c12-bf6d-db6486ab6f57 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:00.167622 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc74bc-f538-4c12-bf6d-db6486ab6f57 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:00.172302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea6994-e732-4039-98d6-ce3ee2cfd94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:00.173190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea6994-e732-4039-98d6-ce3ee2cfd94b admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:00.173476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea6994-e732-4039-98d6-ce3ee2cfd94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:00.173758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea6994-e732-4039-98d6-ce3ee2cfd94b admin admin] PATH_INFO: `/v3/projects/92772c855c974c3e8f58cb3a45fcc7fc/users/7eabfcba288e456fbb416d4c0f8839e2/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:00.178504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:00.197082 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea6994-e732-4039-98d6-ce3ee2cfd94b admin admin] RBAC: Authorizing `identity:create_grant(project_id=92772c855c974c3e8f58cb3a45fcc7fc, user_id=7eabfcba288e456fbb416d4c0f8839e2, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:00.200292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea6994-e732-4039-98d6-ce3ee2cfd94b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:00.329152 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 892/1776] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:00 2022] PUT /identity/v3/projects/92772c855c974c3e8f58cb3a45fcc7fc/users/7eabfcba288e456fbb416d4c0f8839e2/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:00.343672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84f4ac-0861-4860-b9da-e086ff0a4dd0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:00.344278 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84f4ac-0861-4860-b9da-e086ff0a4dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:00.344526 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84f4ac-0861-4860-b9da-e086ff0a4dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:00.387799 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6d84f4ac-0861-4860-b9da-e086ff0a4dd0 None None] MFA Rules not processed for user `7eabfcba288e456fbb416d4c0f8839e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:00.390425 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6d84f4ac-0861-4860-b9da-e086ff0a4dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:00.420891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:00.487504 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 885/1777] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:00.491960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea6994-e732-4039-98d6-ce3ee2cfd94b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:00.571850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea6994-e732-4039-98d6-ce3ee2cfd94b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:00.577313 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27146daf-7ad3-41b6-be5f-1ac2c72bbcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:00.578222 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27146daf-7ad3-41b6-be5f-1ac2c72bbcfa admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:00.578572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27146daf-7ad3-41b6-be5f-1ac2c72bbcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:00.578920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27146daf-7ad3-41b6-be5f-1ac2c72bbcfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:00.579783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27146daf-7ad3-41b6-be5f-1ac2c72bbcfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:00.582423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27146daf-7ad3-41b6-be5f-1ac2c72bbcfa admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:00.676668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 893/1778] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:00 2022] POST /identity/v3/projects => generated 367 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:00.680656 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84f4ac-0861-4860-b9da-e086ff0a4dd0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:00.692817 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84f4ac-0861-4860-b9da-e086ff0a4dd0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:00.754533 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67c975-bafd-46a9-b8ef-57c0354e6e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:00.755268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67c975-bafd-46a9-b8ef-57c0354e6e6e admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:00.755557 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67c975-bafd-46a9-b8ef-57c0354e6e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:00.755828 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67c975-bafd-46a9-b8ef-57c0354e6e6e admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:00.756531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67c975-bafd-46a9-b8ef-57c0354e6e6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:00.758440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67c975-bafd-46a9-b8ef-57c0354e6e6e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:00.869344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 886/1779] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:00 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:00.873353 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27146daf-7ad3-41b6-be5f-1ac2c72bbcfa admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:00.888622 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27146daf-7ad3-41b6-be5f-1ac2c72bbcfa admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:00.893836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc40a7-8d7f-4c13-b6d5-6a401e9ac669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:00.894916 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc40a7-8d7f-4c13-b6d5-6a401e9ac669 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:00.895328 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc40a7-8d7f-4c13-b6d5-6a401e9ac669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:00.895615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc40a7-8d7f-4c13-b6d5-6a401e9ac669 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:00.896403 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc40a7-8d7f-4c13-b6d5-6a401e9ac669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:00.898093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc40a7-8d7f-4c13-b6d5-6a401e9ac669 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:00.909656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 894/1780] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:00.913974 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67c975-bafd-46a9-b8ef-57c0354e6e6e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:00.924425 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67c975-bafd-46a9-b8ef-57c0354e6e6e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:00.929768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57274be8-7aa8-4073-b480-4278bd86fef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:00.930669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57274be8-7aa8-4073-b480-4278bd86fef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:00.930955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57274be8-7aa8-4073-b480-4278bd86fef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:00.931269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57274be8-7aa8-4073-b480-4278bd86fef4 admin admin] PATH_INFO: `/v3/projects/a75e123aeb7a44ca8bd0676487e0aad4/users/7d740ab3c6814a23891bf530e00355dc/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:00.948206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57274be8-7aa8-4073-b480-4278bd86fef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a75e123aeb7a44ca8bd0676487e0aad4, user_id=7d740ab3c6814a23891bf530e00355dc, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:00.950866 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57274be8-7aa8-4073-b480-4278bd86fef4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:01.040065 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 887/1781] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:00 2022] PUT /identity/v3/projects/a75e123aeb7a44ca8bd0676487e0aad4/users/7d740ab3c6814a23891bf530e00355dc/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:01.044123 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc40a7-8d7f-4c13-b6d5-6a401e9ac669 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:01.055021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc40a7-8d7f-4c13-b6d5-6a401e9ac669 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:01.097752 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:01.114871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9f147-ca21-42a2-ac9b-e552ed7b9365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:01.115568 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9f147-ca21-42a2-ac9b-e552ed7b9365 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:01.115855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9f147-ca21-42a2-ac9b-e552ed7b9365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:01.116134 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9f147-ca21-42a2-ac9b-e552ed7b9365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:01.116885 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9f147-ca21-42a2-ac9b-e552ed7b9365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:01.118529 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9f147-ca21-42a2-ac9b-e552ed7b9365 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:01.128428 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 895/1782] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:01.132675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57274be8-7aa8-4073-b480-4278bd86fef4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:01.142839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57274be8-7aa8-4073-b480-4278bd86fef4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:01.147810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4a895-ce0a-473a-9df5-d271c0f7d3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:01.148687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4a895-ce0a-473a-9df5-d271c0f7d3e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:01.148968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4a895-ce0a-473a-9df5-d271c0f7d3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:01.149277 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4a895-ce0a-473a-9df5-d271c0f7d3e4 admin admin] PATH_INFO: `/v3/projects/a75e123aeb7a44ca8bd0676487e0aad4/users/7d740ab3c6814a23891bf530e00355dc/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:01.151230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4a895-ce0a-473a-9df5-d271c0f7d3e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a75e123aeb7a44ca8bd0676487e0aad4, user_id=7d740ab3c6814a23891bf530e00355dc, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:01.153853 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4a895-ce0a-473a-9df5-d271c0f7d3e4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:01.173936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 888/1783] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:01 2022] PUT /identity/v3/projects/a75e123aeb7a44ca8bd0676487e0aad4/users/7d740ab3c6814a23891bf530e00355dc/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:01.178245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6cf0-4e5c-4e3a-b570-60ba0f660555 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:01.178539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6cf0-4e5c-4e3a-b570-60ba0f660555 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:01.178839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6cf0-4e5c-4e3a-b570-60ba0f660555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:01.205178 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e58d6cf0-4e5c-4e3a-b570-60ba0f660555 None None] MFA Rules not processed for user `7d740ab3c6814a23891bf530e00355dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:01.206412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e58d6cf0-4e5c-4e3a-b570-60ba0f660555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:01.258628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cb4f8-456a-40bd-87c8-8027dc7f67c9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:01.259293 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cb4f8-456a-40bd-87c8-8027dc7f67c9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:01.259720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cb4f8-456a-40bd-87c8-8027dc7f67c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:01.279651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 896/1784] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:01 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:01.290896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d6cf0-4e5c-4e3a-b570-60ba0f660555 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:01.291897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f43cb4f8-456a-40bd-87c8-8027dc7f67c9 None None] MFA Rules not processed for user `4ebbf52381904a2b8e8ef8f67d7f3d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:01.293448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f43cb4f8-456a-40bd-87c8-8027dc7f67c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:01.348171 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d6cf0-4e5c-4e3a-b570-60ba0f660555 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:01.352798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27903fe6-585e-4b87-85fa-16484ee4fbef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:01.353506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27903fe6-585e-4b87-85fa-16484ee4fbef service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:01.353788 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27903fe6-585e-4b87-85fa-16484ee4fbef service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:01.354059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27903fe6-585e-4b87-85fa-16484ee4fbef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:01.364362 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27903fe6-585e-4b87-85fa-16484ee4fbef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:01.365796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27903fe6-585e-4b87-85fa-16484ee4fbef service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:01.371167 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 889/1785] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:01 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:01.375907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cb4f8-456a-40bd-87c8-8027dc7f67c9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:01.377174 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f43cb4f8-456a-40bd-87c8-8027dc7f67c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:01.380738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 897/1786] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:19:01.407303 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:01.467243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cb4f8-456a-40bd-87c8-8027dc7f67c9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:01.474122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa81f0-649f-4886-aff5-8e21855c4382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:01.475147 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa81f0-649f-4886-aff5-8e21855c4382 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:01.475804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa81f0-649f-4886-aff5-8e21855c4382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:01.476334 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa81f0-649f-4886-aff5-8e21855c4382 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:01.482543 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa81f0-649f-4886-aff5-8e21855c4382 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:01.484933 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa81f0-649f-4886-aff5-8e21855c4382 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:01.618002 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 890/1787] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:01 2022] POST /identity/v3/projects => generated 405 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:01.619145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27903fe6-585e-4b87-85fa-16484ee4fbef service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:01.631850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27903fe6-585e-4b87-85fa-16484ee4fbef service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:01.707967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6954d-b4c2-46f5-8ec7-d8d1daf8631a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:01.708853 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6954d-b4c2-46f5-8ec7-d8d1daf8631a admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:01.709285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6954d-b4c2-46f5-8ec7-d8d1daf8631a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:01.709571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6954d-b4c2-46f5-8ec7-d8d1daf8631a admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:01.710292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6954d-b4c2-46f5-8ec7-d8d1daf8631a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:01.712478 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6954d-b4c2-46f5-8ec7-d8d1daf8631a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:01.722746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:01.808841 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 898/1788] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:01 2022] POST /identity/v3/users => generated 309 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:01.811453 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa81f0-649f-4886-aff5-8e21855c4382 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:01.823929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa81f0-649f-4886-aff5-8e21855c4382 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:01.828751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1446e-5ca9-4a51-9491-b418913347ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:01.829469 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1446e-5ca9-4a51-9491-b418913347ad admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:01.829763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1446e-5ca9-4a51-9491-b418913347ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:01.837920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1446e-5ca9-4a51-9491-b418913347ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:01.838725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1446e-5ca9-4a51-9491-b418913347ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:01.840386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1446e-5ca9-4a51-9491-b418913347ad admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:01.853237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 891/1789] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:01.857894 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6954d-b4c2-46f5-8ec7-d8d1daf8631a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:01.874883 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6954d-b4c2-46f5-8ec7-d8d1daf8631a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:01.882156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8b0ae-3422-4d11-b1b3-2b4708d63886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:01.883802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8b0ae-3422-4d11-b1b3-2b4708d63886 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:01.884371 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8b0ae-3422-4d11-b1b3-2b4708d63886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:01.885235 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8b0ae-3422-4d11-b1b3-2b4708d63886 admin admin] PATH_INFO: `/v3/projects/b5196786415d4947acac6a3e11ca1da8/users/bcac9d6b8f4840b59fea43f00fa71e86/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:01.910633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8b0ae-3422-4d11-b1b3-2b4708d63886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5196786415d4947acac6a3e11ca1da8, user_id=bcac9d6b8f4840b59fea43f00fa71e86, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:01.915341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8b0ae-3422-4d11-b1b3-2b4708d63886 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:01.982918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 899/1790] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:01 2022] PUT /identity/v3/projects/b5196786415d4947acac6a3e11ca1da8/users/bcac9d6b8f4840b59fea43f00fa71e86/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:01.986757 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1446e-5ca9-4a51-9491-b418913347ad admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:01.997177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1446e-5ca9-4a51-9491-b418913347ad admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:02.109852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a81d79-6f56-4cba-87e8-b7e384577ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:02.110605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a81d79-6f56-4cba-87e8-b7e384577ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:02.111023 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a81d79-6f56-4cba-87e8-b7e384577ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:02.111645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a81d79-6f56-4cba-87e8-b7e384577ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:02.114628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a81d79-6f56-4cba-87e8-b7e384577ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:02.115892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a81d79-6f56-4cba-87e8-b7e384577ec8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:02.134030 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 892/1791] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:02.138073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8b0ae-3422-4d11-b1b3-2b4708d63886 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:02.157375 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8b0ae-3422-4d11-b1b3-2b4708d63886 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:02.160812 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103de9f-44c2-42c0-a74b-7c31ddff8cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:02.161719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103de9f-44c2-42c0-a74b-7c31ddff8cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:02.162002 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103de9f-44c2-42c0-a74b-7c31ddff8cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:02.162313 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103de9f-44c2-42c0-a74b-7c31ddff8cb2 admin admin] PATH_INFO: `/v3/projects/b5196786415d4947acac6a3e11ca1da8/users/bcac9d6b8f4840b59fea43f00fa71e86/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:02.164457 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103de9f-44c2-42c0-a74b-7c31ddff8cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5196786415d4947acac6a3e11ca1da8, user_id=bcac9d6b8f4840b59fea43f00fa71e86, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:02.167852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103de9f-44c2-42c0-a74b-7c31ddff8cb2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:02.181642 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 900/1792] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:02 2022] PUT /identity/v3/projects/b5196786415d4947acac6a3e11ca1da8/users/bcac9d6b8f4840b59fea43f00fa71e86/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:03.904525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307864f0-1d58-4f6e-bdbd-8a5dee75840f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:03.905422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307864f0-1d58-4f6e-bdbd-8a5dee75840f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:03.905710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307864f0-1d58-4f6e-bdbd-8a5dee75840f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:03.954883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-307864f0-1d58-4f6e-bdbd-8a5dee75840f None None] MFA Rules not processed for user `bcac9d6b8f4840b59fea43f00fa71e86`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:03.957618 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-307864f0-1d58-4f6e-bdbd-8a5dee75840f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:04.100640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 893/1793] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:03 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:05.258798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721f892-0c7b-45c6-ae0c-110e372e4f36 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:05.259967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721f892-0c7b-45c6-ae0c-110e372e4f36 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:05.260561 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721f892-0c7b-45c6-ae0c-110e372e4f36 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:05.262474 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 901/1794] 10.222.0.67 () {54 vars in 921 bytes} [Fri Jun 24 15:19:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:19:05.271100 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26248605-d753-4dc3-95ca-73c8ad234bde None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:05.272840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26248605-d753-4dc3-95ca-73c8ad234bde None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:05.273331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26248605-d753-4dc3-95ca-73c8ad234bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:05.282558 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:05.330105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-26248605-d753-4dc3-95ca-73c8ad234bde None None] MFA Rules not processed for user `f612d6ff06fc458e8c1178cbf8321f79`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:05.335858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-26248605-d753-4dc3-95ca-73c8ad234bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:05.429146 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 894/1795] 10.222.0.67 () {58 vars in 1018 bytes} [Fri Jun 24 15:19:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:05.443109 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721f892-0c7b-45c6-ae0c-110e372e4f36 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:05.450268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f721f892-0c7b-45c6-ae0c-110e372e4f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:05.476799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721f892-0c7b-45c6-ae0c-110e372e4f36 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:05.488854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b4ce9-a81f-435a-8795-73a721528a6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:05.490564 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b4ce9-a81f-435a-8795-73a721528a6a service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:05.490564 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b4ce9-a81f-435a-8795-73a721528a6a service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:05.490564 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b4ce9-a81f-435a-8795-73a721528a6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:05.513518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b4ce9-a81f-435a-8795-73a721528a6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:05.515370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b4ce9-a81f-435a-8795-73a721528a6a service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:05.529452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 902/1796] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:05.810528 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26248605-d753-4dc3-95ca-73c8ad234bde None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:05.891559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26248605-d753-4dc3-95ca-73c8ad234bde None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:05.896575 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1932280-fd8d-448d-bf7e-4555ec3b3ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:05.897480 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1932280-fd8d-448d-bf7e-4555ec3b3ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:05.897761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1932280-fd8d-448d-bf7e-4555ec3b3ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:05.898035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1932280-fd8d-448d-bf7e-4555ec3b3ea2 admin admin] PATH_INFO: `/v3/users/7eabfcba288e456fbb416d4c0f8839e2` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:05.899241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1932280-fd8d-448d-bf7e-4555ec3b3ea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eabfcba288e456fbb416d4c0f8839e2)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:05.901219 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1932280-fd8d-448d-bf7e-4555ec3b3ea2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:06.105411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d1932280-fd8d-448d-bf7e-4555ec3b3ea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eabfcba288e456fbb416d4c0f8839e2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:06.124775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d1932280-fd8d-448d-bf7e-4555ec3b3ea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eabfcba288e456fbb416d4c0f8839e2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:06.210802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d1932280-fd8d-448d-bf7e-4555ec3b3ea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eabfcba288e456fbb416d4c0f8839e2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:06.211580 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d1932280-fd8d-448d-bf7e-4555ec3b3ea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eabfcba288e456fbb416d4c0f8839e2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:06.221606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d1932280-fd8d-448d-bf7e-4555ec3b3ea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eabfcba288e456fbb416d4c0f8839e2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:06.230860 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 895/1797] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:05 2022] DELETE /identity/v3/users/7eabfcba288e456fbb416d4c0f8839e2 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:06.235432 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566f011-eccb-4939-9002-38a84adafeca None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:06.236184 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566f011-eccb-4939-9002-38a84adafeca None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:06.236766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566f011-eccb-4939-9002-38a84adafeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:06.258808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9566f011-eccb-4939-9002-38a84adafeca None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:06.260342 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9566f011-eccb-4939-9002-38a84adafeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:06.373697 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 903/1798] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:06.467032 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1932280-fd8d-448d-bf7e-4555ec3b3ea2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:06.468531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d1932280-fd8d-448d-bf7e-4555ec3b3ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:06.495425 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:06.554530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1932280-fd8d-448d-bf7e-4555ec3b3ea2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:06.559450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30473f50-e5af-417e-a165-56fb8fa49efb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:06.560252 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30473f50-e5af-417e-a165-56fb8fa49efb service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:06.561343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30473f50-e5af-417e-a165-56fb8fa49efb service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:06.561635 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30473f50-e5af-417e-a165-56fb8fa49efb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:06.577785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30473f50-e5af-417e-a165-56fb8fa49efb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:06.586990 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30473f50-e5af-417e-a165-56fb8fa49efb service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:06.594622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 896/1799] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:06.609898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75abbd2-1365-4a77-8921-4803b6a9d7a2 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:06.609898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75abbd2-1365-4a77-8921-4803b6a9d7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:06.609898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75abbd2-1365-4a77-8921-4803b6a9d7a2 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:06.609898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 904/1800] 10.222.0.67 () {54 vars in 921 bytes} [Fri Jun 24 15:19:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:19:06.636519 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386596f3-6ec4-426c-a0bc-038c0fb9e3d4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:06.639026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386596f3-6ec4-426c-a0bc-038c0fb9e3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:06.641150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386596f3-6ec4-426c-a0bc-038c0fb9e3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:06.677583 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-386596f3-6ec4-426c-a0bc-038c0fb9e3d4 None None] MFA Rules not processed for user `f612d6ff06fc458e8c1178cbf8321f79`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:06.678928 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-386596f3-6ec4-426c-a0bc-038c0fb9e3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:06.750779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 897/1801] 10.222.0.67 () {58 vars in 1018 bytes} [Fri Jun 24 15:19:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:06.766079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75abbd2-1365-4a77-8921-4803b6a9d7a2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:06.770267 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e75abbd2-1365-4a77-8921-4803b6a9d7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:06.789064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75abbd2-1365-4a77-8921-4803b6a9d7a2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:06.796785 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b73f13-b3fa-41a3-8821-f3003b43493e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:06.799527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b73f13-b3fa-41a3-8821-f3003b43493e service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:06.799527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b73f13-b3fa-41a3-8821-f3003b43493e service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:06.799527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b73f13-b3fa-41a3-8821-f3003b43493e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:06.808900 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b73f13-b3fa-41a3-8821-f3003b43493e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:06.811528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b73f13-b3fa-41a3-8821-f3003b43493e service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:06.822185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386596f3-6ec4-426c-a0bc-038c0fb9e3d4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:06.823774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-386596f3-6ec4-426c-a0bc-038c0fb9e3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:06.833213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 905/1802] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:19:06.840697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386596f3-6ec4-426c-a0bc-038c0fb9e3d4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:06.847839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74407fcd-4d97-4094-b154-2ab7a7d434f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:06.848734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74407fcd-4d97-4094-b154-2ab7a7d434f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:06.849011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74407fcd-4d97-4094-b154-2ab7a7d434f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:06.849280 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74407fcd-4d97-4094-b154-2ab7a7d434f6 admin admin] PATH_INFO: `/v3/users/4ebbf52381904a2b8e8ef8f67d7f3d3a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:06.850621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74407fcd-4d97-4094-b154-2ab7a7d434f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ebbf52381904a2b8e8ef8f67d7f3d3a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:06.852773 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74407fcd-4d97-4094-b154-2ab7a7d434f6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:07.064537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-74407fcd-4d97-4094-b154-2ab7a7d434f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ebbf52381904a2b8e8ef8f67d7f3d3a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:07.079050 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-74407fcd-4d97-4094-b154-2ab7a7d434f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ebbf52381904a2b8e8ef8f67d7f3d3a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:07.142205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-74407fcd-4d97-4094-b154-2ab7a7d434f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ebbf52381904a2b8e8ef8f67d7f3d3a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:07.143085 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-74407fcd-4d97-4094-b154-2ab7a7d434f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ebbf52381904a2b8e8ef8f67d7f3d3a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:07.153556 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-74407fcd-4d97-4094-b154-2ab7a7d434f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ebbf52381904a2b8e8ef8f67d7f3d3a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:07.161793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 898/1803] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:06 2022] DELETE /identity/v3/users/4ebbf52381904a2b8e8ef8f67d7f3d3a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:07.531654 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b73f13-b3fa-41a3-8821-f3003b43493e service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:07.550328 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b73f13-b3fa-41a3-8821-f3003b43493e service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:07.558735 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74407fcd-4d97-4094-b154-2ab7a7d434f6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:07.560802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-74407fcd-4d97-4094-b154-2ab7a7d434f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:07.668236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada5872-fc78-40e7-b3b4-941acd2f9d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:07.669172 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada5872-fc78-40e7-b3b4-941acd2f9d71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:07.669615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada5872-fc78-40e7-b3b4-941acd2f9d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:07.670056 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada5872-fc78-40e7-b3b4-941acd2f9d71 admin admin] PATH_INFO: `/v3/projects/3ec0d7bcce024fbf94e21ccfcb936d7e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:07.672945 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada5872-fc78-40e7-b3b4-941acd2f9d71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ec0d7bcce024fbf94e21ccfcb936d7e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:07.675576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada5872-fc78-40e7-b3b4-941acd2f9d71 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:07.704749 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74407fcd-4d97-4094-b154-2ab7a7d434f6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:07.711717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338e46c-ba70-47fb-8f30-1279f4c3c3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:07.712606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338e46c-ba70-47fb-8f30-1279f4c3c3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:07.712989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338e46c-ba70-47fb-8f30-1279f4c3c3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:07.713275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338e46c-ba70-47fb-8f30-1279f4c3c3db admin admin] PATH_INFO: `/v3/projects/92772c855c974c3e8f58cb3a45fcc7fc` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:07.714649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338e46c-ba70-47fb-8f30-1279f4c3c3db admin admin] RBAC: Authorizing `identity:delete_project(project_id=92772c855c974c3e8f58cb3a45fcc7fc)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:07.716611 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338e46c-ba70-47fb-8f30-1279f4c3c3db admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:07.818518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:07.827725 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cada5872-fc78-40e7-b3b4-941acd2f9d71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec0d7bcce024fbf94e21ccfcb936d7e', 'request_id': 'req-cada5872-fc78-40e7-b3b4-941acd2f9d71'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:07.834384 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cada5872-fc78-40e7-b3b4-941acd2f9d71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec0d7bcce024fbf94e21ccfcb936d7e', 'request_id': 'req-cada5872-fc78-40e7-b3b4-941acd2f9d71'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:07.891505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1338e46c-ba70-47fb-8f30-1279f4c3c3db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92772c855c974c3e8f58cb3a45fcc7fc', 'request_id': 'req-1338e46c-ba70-47fb-8f30-1279f4c3c3db'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:07.898491 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1338e46c-ba70-47fb-8f30-1279f4c3c3db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92772c855c974c3e8f58cb3a45fcc7fc', 'request_id': 'req-1338e46c-ba70-47fb-8f30-1279f4c3c3db'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:07.942836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cada5872-fc78-40e7-b3b4-941acd2f9d71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec0d7bcce024fbf94e21ccfcb936d7e', 'request_id': 'req-cada5872-fc78-40e7-b3b4-941acd2f9d71'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:07.955458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cada5872-fc78-40e7-b3b4-941acd2f9d71 admin admin] The token cache is being invalidate because project 3ec0d7bcce024fbf94e21ccfcb936d7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:07.955746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cada5872-fc78-40e7-b3b4-941acd2f9d71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:07.956510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cada5872-fc78-40e7-b3b4-941acd2f9d71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:07.959082 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 906/1804] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:07 2022] DELETE /identity/v3/projects/3ec0d7bcce024fbf94e21ccfcb936d7e => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:07.962892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada5872-fc78-40e7-b3b4-941acd2f9d71 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:07.964431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-cada5872-fc78-40e7-b3b4-941acd2f9d71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:08.016942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1338e46c-ba70-47fb-8f30-1279f4c3c3db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92772c855c974c3e8f58cb3a45fcc7fc', 'request_id': 'req-1338e46c-ba70-47fb-8f30-1279f4c3c3db'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.037600 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1338e46c-ba70-47fb-8f30-1279f4c3c3db admin admin] The token cache is being invalidate because project 92772c855c974c3e8f58cb3a45fcc7fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:08.038159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1338e46c-ba70-47fb-8f30-1279f4c3c3db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.039051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1338e46c-ba70-47fb-8f30-1279f4c3c3db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.041424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada5872-fc78-40e7-b3b4-941acd2f9d71 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:08.044565 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 899/1805] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:07 2022] DELETE /identity/v3/projects/92772c855c974c3e8f58cb3a45fcc7fc => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:08.045659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338e46c-ba70-47fb-8f30-1279f4c3c3db admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:08.047306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1338e46c-ba70-47fb-8f30-1279f4c3c3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:08.107612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6a98d-f192-48a3-9c68-8b67281afb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:08.115998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6a98d-f192-48a3-9c68-8b67281afb88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:08.115998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6a98d-f192-48a3-9c68-8b67281afb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:08.115998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6a98d-f192-48a3-9c68-8b67281afb88 admin admin] PATH_INFO: `/v3/users/bcac9d6b8f4840b59fea43f00fa71e86` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:08.116297 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6a98d-f192-48a3-9c68-8b67281afb88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcac9d6b8f4840b59fea43f00fa71e86)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:08.118654 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6a98d-f192-48a3-9c68-8b67281afb88 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:08.134459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338e46c-ba70-47fb-8f30-1279f4c3c3db admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:08.140266 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd7217-5a0a-432f-bb63-ca4e53089811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:08.141160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd7217-5a0a-432f-bb63-ca4e53089811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:08.141478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd7217-5a0a-432f-bb63-ca4e53089811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:08.144983 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd7217-5a0a-432f-bb63-ca4e53089811 admin admin] PATH_INFO: `/v3/users/7d740ab3c6814a23891bf530e00355dc` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:08.144983 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd7217-5a0a-432f-bb63-ca4e53089811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d740ab3c6814a23891bf530e00355dc)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:08.145833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd7217-5a0a-432f-bb63-ca4e53089811 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:08.254173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:08.321798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-69f6a98d-f192-48a3-9c68-8b67281afb88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcac9d6b8f4840b59fea43f00fa71e86'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.333718 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-69f6a98d-f192-48a3-9c68-8b67281afb88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcac9d6b8f4840b59fea43f00fa71e86'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.334413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-69f6a98d-f192-48a3-9c68-8b67281afb88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcac9d6b8f4840b59fea43f00fa71e86'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.341278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-69f6a98d-f192-48a3-9c68-8b67281afb88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcac9d6b8f4840b59fea43f00fa71e86'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.346417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-69f6a98d-f192-48a3-9c68-8b67281afb88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcac9d6b8f4840b59fea43f00fa71e86'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.352999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eccd7217-5a0a-432f-bb63-ca4e53089811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d740ab3c6814a23891bf530e00355dc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.365262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eccd7217-5a0a-432f-bb63-ca4e53089811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d740ab3c6814a23891bf530e00355dc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.456458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 907/1806] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:07 2022] DELETE /identity/v3/users/bcac9d6b8f4840b59fea43f00fa71e86 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:08.457284 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6a98d-f192-48a3-9c68-8b67281afb88 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:08.458603 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-69f6a98d-f192-48a3-9c68-8b67281afb88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:08.503607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eccd7217-5a0a-432f-bb63-ca4e53089811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d740ab3c6814a23891bf530e00355dc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.503802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eccd7217-5a0a-432f-bb63-ca4e53089811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d740ab3c6814a23891bf530e00355dc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.512363 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eccd7217-5a0a-432f-bb63-ca4e53089811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d740ab3c6814a23891bf530e00355dc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.519472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 900/1807] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:08 2022] DELETE /identity/v3/users/7d740ab3c6814a23891bf530e00355dc => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:08.525143 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6a98d-f192-48a3-9c68-8b67281afb88 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:08.588049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3a2d0-249e-44af-9f0b-ee5e1957ee79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:08.588968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3a2d0-249e-44af-9f0b-ee5e1957ee79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:08.589263 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3a2d0-249e-44af-9f0b-ee5e1957ee79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:08.589534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3a2d0-249e-44af-9f0b-ee5e1957ee79 admin admin] PATH_INFO: `/v3/users/9d2a85c1f30c4ecb9d342495c08d3d17` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:08.591149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3a2d0-249e-44af-9f0b-ee5e1957ee79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d2a85c1f30c4ecb9d342495c08d3d17)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:08.593797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3a2d0-249e-44af-9f0b-ee5e1957ee79 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:08.709527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd7217-5a0a-432f-bb63-ca4e53089811 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:08.711689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-eccd7217-5a0a-432f-bb63-ca4e53089811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:08.726271 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd7217-5a0a-432f-bb63-ca4e53089811 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:08.732261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d59cc-9a35-42a8-9223-f40be9af4bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:08.733152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d59cc-9a35-42a8-9223-f40be9af4bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:08.742343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d59cc-9a35-42a8-9223-f40be9af4bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:08.742343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d59cc-9a35-42a8-9223-f40be9af4bee admin admin] PATH_INFO: `/v3/projects/b5196786415d4947acac6a3e11ca1da8` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:08.742343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d59cc-9a35-42a8-9223-f40be9af4bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5196786415d4947acac6a3e11ca1da8)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:08.742343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d59cc-9a35-42a8-9223-f40be9af4bee admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:08.769335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d4a3a2d0-249e-44af-9f0b-ee5e1957ee79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d2a85c1f30c4ecb9d342495c08d3d17'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.782685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d4a3a2d0-249e-44af-9f0b-ee5e1957ee79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d2a85c1f30c4ecb9d342495c08d3d17'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.783037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d4a3a2d0-249e-44af-9f0b-ee5e1957ee79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d2a85c1f30c4ecb9d342495c08d3d17'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.790706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d4a3a2d0-249e-44af-9f0b-ee5e1957ee79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d2a85c1f30c4ecb9d342495c08d3d17'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.791442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d4a3a2d0-249e-44af-9f0b-ee5e1957ee79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d2a85c1f30c4ecb9d342495c08d3d17'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.829314 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa7d59cc-9a35-42a8-9223-f40be9af4bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5196786415d4947acac6a3e11ca1da8', 'request_id': 'req-aa7d59cc-9a35-42a8-9223-f40be9af4bee'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.836993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa7d59cc-9a35-42a8-9223-f40be9af4bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5196786415d4947acac6a3e11ca1da8', 'request_id': 'req-aa7d59cc-9a35-42a8-9223-f40be9af4bee'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.844945 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 908/1808] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:08 2022] DELETE /identity/v3/users/9d2a85c1f30c4ecb9d342495c08d3d17 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:08.850092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0edb6a-977e-455f-ab10-a63db00ce942 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:08.850538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0edb6a-977e-455f-ab10-a63db00ce942 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:08.850860 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0edb6a-977e-455f-ab10-a63db00ce942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:08.869196 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7f0edb6a-977e-455f-ab10-a63db00ce942 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:08.870408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7f0edb6a-977e-455f-ab10-a63db00ce942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:08.890444 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa7d59cc-9a35-42a8-9223-f40be9af4bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5196786415d4947acac6a3e11ca1da8', 'request_id': 'req-aa7d59cc-9a35-42a8-9223-f40be9af4bee'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.902087 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa7d59cc-9a35-42a8-9223-f40be9af4bee admin admin] The token cache is being invalidate because project b5196786415d4947acac6a3e11ca1da8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:08.902472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa7d59cc-9a35-42a8-9223-f40be9af4bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.903205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa7d59cc-9a35-42a8-9223-f40be9af4bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:08.904881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 901/1809] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:08 2022] DELETE /identity/v3/projects/b5196786415d4947acac6a3e11ca1da8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:08.909842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964f0ec-d419-4e65-8fb1-62e9d5df8604 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:08.910204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964f0ec-d419-4e65-8fb1-62e9d5df8604 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:08.910558 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964f0ec-d419-4e65-8fb1-62e9d5df8604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:08.928927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e964f0ec-d419-4e65-8fb1-62e9d5df8604 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:08.929427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 909/1810] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:08.930868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e964f0ec-d419-4e65-8fb1-62e9d5df8604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:08.938770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0edb6a-977e-455f-ab10-a63db00ce942 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:08.939603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 902/1811] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:08.943382 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7f0edb6a-977e-455f-ab10-a63db00ce942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:08.947991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb477a9c-0b48-4226-b85b-811f8b10ce58 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:08.953003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb477a9c-0b48-4226-b85b-811f8b10ce58 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:08.953003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb477a9c-0b48-4226-b85b-811f8b10ce58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:08.966352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bb477a9c-0b48-4226-b85b-811f8b10ce58 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:08.967593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bb477a9c-0b48-4226-b85b-811f8b10ce58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:08.976242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 903/1812] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:08.977160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb477a9c-0b48-4226-b85b-811f8b10ce58 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:08.978493 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bb477a9c-0b48-4226-b85b-811f8b10ce58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:08.991063 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb477a9c-0b48-4226-b85b-811f8b10ce58 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:08.996124 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdbb5c-20ae-411f-941d-f6cc9c1ee02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:08.997206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdbb5c-20ae-411f-941d-f6cc9c1ee02a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:08.999062 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdbb5c-20ae-411f-941d-f6cc9c1ee02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:08.999062 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdbb5c-20ae-411f-941d-f6cc9c1ee02a admin admin] PATH_INFO: `/v3/projects/a75e123aeb7a44ca8bd0676487e0aad4` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:08.999408 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fdbb5c-20ae-411f-941d-f6cc9c1ee02a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a75e123aeb7a44ca8bd0676487e0aad4)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:09.002292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fdbb5c-20ae-411f-941d-f6cc9c1ee02a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:09.007296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0edb6a-977e-455f-ab10-a63db00ce942 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:09.011661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58066d26-e979-4770-b6a0-d6736d72ee61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:09.012316 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58066d26-e979-4770-b6a0-d6736d72ee61 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:09.012596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58066d26-e979-4770-b6a0-d6736d72ee61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:09.012869 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58066d26-e979-4770-b6a0-d6736d72ee61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:09.022561 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58066d26-e979-4770-b6a0-d6736d72ee61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:09.027761 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58066d26-e979-4770-b6a0-d6736d72ee61 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:09.033906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 910/1813] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:19:09.034777 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58066d26-e979-4770-b6a0-d6736d72ee61 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:09.043874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58066d26-e979-4770-b6a0-d6736d72ee61 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:09.048467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef6e2a-2022-4e89-967e-03b6810795cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:09.049114 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef6e2a-2022-4e89-967e-03b6810795cb admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:09.050732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef6e2a-2022-4e89-967e-03b6810795cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:09.050732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef6e2a-2022-4e89-967e-03b6810795cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:09.051358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef6e2a-2022-4e89-967e-03b6810795cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:19:09.051358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef6e2a-2022-4e89-967e-03b6810795cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:09.052905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef6e2a-2022-4e89-967e-03b6810795cb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:09.065194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 911/1814] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:19:09.070261 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef6e2a-2022-4e89-967e-03b6810795cb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:09.080114 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef6e2a-2022-4e89-967e-03b6810795cb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:09.085300 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d016e-7371-4691-9311-cb99418f5ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:09.086391 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d016e-7371-4691-9311-cb99418f5ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:09.086678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d016e-7371-4691-9311-cb99418f5ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:09.086956 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d016e-7371-4691-9311-cb99418f5ec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:09.087686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d016e-7371-4691-9311-cb99418f5ec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:09.090056 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d016e-7371-4691-9311-cb99418f5ec4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:09.114183 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-04fdbb5c-20ae-411f-941d-f6cc9c1ee02a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a75e123aeb7a44ca8bd0676487e0aad4', 'request_id': 'req-04fdbb5c-20ae-411f-941d-f6cc9c1ee02a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:09.120506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-04fdbb5c-20ae-411f-941d-f6cc9c1ee02a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a75e123aeb7a44ca8bd0676487e0aad4', 'request_id': 'req-04fdbb5c-20ae-411f-941d-f6cc9c1ee02a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:09.216805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-04fdbb5c-20ae-411f-941d-f6cc9c1ee02a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a75e123aeb7a44ca8bd0676487e0aad4', 'request_id': 'req-04fdbb5c-20ae-411f-941d-f6cc9c1ee02a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:09.230404 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-04fdbb5c-20ae-411f-941d-f6cc9c1ee02a admin admin] The token cache is being invalidate because project a75e123aeb7a44ca8bd0676487e0aad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:09.230632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-04fdbb5c-20ae-411f-941d-f6cc9c1ee02a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:09.231635 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-04fdbb5c-20ae-411f-941d-f6cc9c1ee02a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:09.233198 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 904/1815] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:08 2022] DELETE /identity/v3/projects/a75e123aeb7a44ca8bd0676487e0aad4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:09.238590 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf7384-75c6-48f1-81b1-450e0021c81a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:09.239352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf7384-75c6-48f1-81b1-450e0021c81a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:09.239797 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf7384-75c6-48f1-81b1-450e0021c81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:09.256910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 912/1816] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:09 2022] POST /identity/v3/projects => generated 371 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:09.259668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d016e-7371-4691-9311-cb99418f5ec4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:09.260079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3e7d016e-7371-4691-9311-cb99418f5ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:09.265118 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4eaf7384-75c6-48f1-81b1-450e0021c81a None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:09.266990 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4eaf7384-75c6-48f1-81b1-450e0021c81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:09.309338 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:09.400011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 905/1817] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:09.402927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352fc0f-e37c-4ca1-a5d3-6b46a824cf50 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:09.402927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352fc0f-e37c-4ca1-a5d3-6b46a824cf50 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:09.402927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352fc0f-e37c-4ca1-a5d3-6b46a824cf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:09.405314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d016e-7371-4691-9311-cb99418f5ec4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:09.419869 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34551e-027f-4407-a6e6-e6d136b22178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:09.420720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34551e-027f-4407-a6e6-e6d136b22178 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:09.422378 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34551e-027f-4407-a6e6-e6d136b22178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:09.423211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34551e-027f-4407-a6e6-e6d136b22178 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:09.423329 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e352fc0f-e37c-4ca1-a5d3-6b46a824cf50 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:09.424754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e352fc0f-e37c-4ca1-a5d3-6b46a824cf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:09.425009 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34551e-027f-4407-a6e6-e6d136b22178 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:09.427645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34551e-027f-4407-a6e6-e6d136b22178 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:09.437981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 906/1818] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:09.438895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352fc0f-e37c-4ca1-a5d3-6b46a824cf50 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:09.449533 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352fc0f-e37c-4ca1-a5d3-6b46a824cf50 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:09.455640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4aa47-fd31-4864-be1c-65b9644d0605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:09.456604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4aa47-fd31-4864-be1c-65b9644d0605 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:09.456990 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4aa47-fd31-4864-be1c-65b9644d0605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:09.459527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4aa47-fd31-4864-be1c-65b9644d0605 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:09.459527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4aa47-fd31-4864-be1c-65b9644d0605 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:19:09.459527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4aa47-fd31-4864-be1c-65b9644d0605 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:09.460510 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4aa47-fd31-4864-be1c-65b9644d0605 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:09.475277 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 907/1819] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:19:09.477792 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4aa47-fd31-4864-be1c-65b9644d0605 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:09.489797 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4aa47-fd31-4864-be1c-65b9644d0605 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:09.494596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b84df-40a0-4e22-b4ee-5c0dad6436ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:09.495301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b84df-40a0-4e22-b4ee-5c0dad6436ba admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:09.495656 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b84df-40a0-4e22-b4ee-5c0dad6436ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:09.496007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b84df-40a0-4e22-b4ee-5c0dad6436ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:09.496770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b84df-40a0-4e22-b4ee-5c0dad6436ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:09.498680 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b84df-40a0-4e22-b4ee-5c0dad6436ba admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:09.518226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 913/1820] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:09 2022] POST /identity/v3/users => generated 286 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:09.519004 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34551e-027f-4407-a6e6-e6d136b22178 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:09.529291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34551e-027f-4407-a6e6-e6d136b22178 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:09.535331 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980461d2-6515-4984-b9e6-1565b85ec1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:09.536148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980461d2-6515-4984-b9e6-1565b85ec1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:09.536446 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980461d2-6515-4984-b9e6-1565b85ec1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:09.536744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980461d2-6515-4984-b9e6-1565b85ec1c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:09.540621 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980461d2-6515-4984-b9e6-1565b85ec1c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:09.540621 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980461d2-6515-4984-b9e6-1565b85ec1c6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:09.555383 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 914/1821] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:19:09.556906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980461d2-6515-4984-b9e6-1565b85ec1c6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:09.559047 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-980461d2-6515-4984-b9e6-1565b85ec1c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:09.572594 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980461d2-6515-4984-b9e6-1565b85ec1c6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:09.577816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed303342-b26e-47fa-bf34-0226d59dc028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:09.578699 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed303342-b26e-47fa-bf34-0226d59dc028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:09.579160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed303342-b26e-47fa-bf34-0226d59dc028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:09.579472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed303342-b26e-47fa-bf34-0226d59dc028 admin admin] PATH_INFO: `/v3/projects/1be816b1d9264d72804cb89fac91ae48` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:09.583641 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed303342-b26e-47fa-bf34-0226d59dc028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1be816b1d9264d72804cb89fac91ae48)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:09.587084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed303342-b26e-47fa-bf34-0226d59dc028 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:09.597942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 908/1822] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:09 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:09.597942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b84df-40a0-4e22-b4ee-5c0dad6436ba admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:09.666750 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b84df-40a0-4e22-b4ee-5c0dad6436ba admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:09.671712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d20de-fafd-4cfb-a0ce-4a8568839097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:09.672630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d20de-fafd-4cfb-a0ce-4a8568839097 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:09.672911 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d20de-fafd-4cfb-a0ce-4a8568839097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:09.673184 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d20de-fafd-4cfb-a0ce-4a8568839097 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:09.673900 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d20de-fafd-4cfb-a0ce-4a8568839097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:09.675558 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d20de-fafd-4cfb-a0ce-4a8568839097 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:09.685167 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 909/1823] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:09.690489 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d20de-fafd-4cfb-a0ce-4a8568839097 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:09.695415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d20de-fafd-4cfb-a0ce-4a8568839097 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:09.700393 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70e523-01c8-4ff9-bf11-a81d8b8527f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:09.700959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70e523-01c8-4ff9-bf11-a81d8b8527f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:09.701375 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70e523-01c8-4ff9-bf11-a81d8b8527f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:09.701660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70e523-01c8-4ff9-bf11-a81d8b8527f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:09.702347 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70e523-01c8-4ff9-bf11-a81d8b8527f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:09.704357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70e523-01c8-4ff9-bf11-a81d8b8527f6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:09.720180 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ed303342-b26e-47fa-bf34-0226d59dc028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1be816b1d9264d72804cb89fac91ae48', 'request_id': 'req-ed303342-b26e-47fa-bf34-0226d59dc028'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:09.728166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ed303342-b26e-47fa-bf34-0226d59dc028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1be816b1d9264d72804cb89fac91ae48', 'request_id': 'req-ed303342-b26e-47fa-bf34-0226d59dc028'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:09.765114 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:09.787603 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ed303342-b26e-47fa-bf34-0226d59dc028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1be816b1d9264d72804cb89fac91ae48', 'request_id': 'req-ed303342-b26e-47fa-bf34-0226d59dc028'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:09.793664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 910/1824] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:09 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:09.794532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70e523-01c8-4ff9-bf11-a81d8b8527f6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:09.799274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ed303342-b26e-47fa-bf34-0226d59dc028 admin admin] The token cache is being invalidate because project 1be816b1d9264d72804cb89fac91ae48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:09.799571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ed303342-b26e-47fa-bf34-0226d59dc028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:09.800439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ed303342-b26e-47fa-bf34-0226d59dc028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:09.802138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 915/1825] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:09 2022] DELETE /identity/v3/projects/1be816b1d9264d72804cb89fac91ae48 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:09.802966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed303342-b26e-47fa-bf34-0226d59dc028 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:09.804283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ed303342-b26e-47fa-bf34-0226d59dc028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:09.805031 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70e523-01c8-4ff9-bf11-a81d8b8527f6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:09.871865 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19943184-4c57-4ee4-a114-95f1debccf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:09.872766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19943184-4c57-4ee4-a114-95f1debccf80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:09.873053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19943184-4c57-4ee4-a114-95f1debccf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:09.873244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed303342-b26e-47fa-bf34-0226d59dc028 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:09.873349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19943184-4c57-4ee4-a114-95f1debccf80 admin admin] PATH_INFO: `/v3/projects/f1046fd13d5343c491f716f6f5a71c93/users/fad6c305398f44299afa208a2aa36711/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:09.878346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213979d-1d3f-4a6e-8aa1-835fd574d6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:09.882017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213979d-1d3f-4a6e-8aa1-835fd574d6aa admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:09.882017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213979d-1d3f-4a6e-8aa1-835fd574d6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:09.882017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213979d-1d3f-4a6e-8aa1-835fd574d6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:09.882017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213979d-1d3f-4a6e-8aa1-835fd574d6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:09.882458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213979d-1d3f-4a6e-8aa1-835fd574d6aa admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:09.888534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19943184-4c57-4ee4-a114-95f1debccf80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1046fd13d5343c491f716f6f5a71c93, user_id=fad6c305398f44299afa208a2aa36711, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:09.891311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19943184-4c57-4ee4-a114-95f1debccf80 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:09.892181 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 916/1826] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:09.892988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213979d-1d3f-4a6e-8aa1-835fd574d6aa admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:09.897290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7213979d-1d3f-4a6e-8aa1-835fd574d6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:09.910558 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213979d-1d3f-4a6e-8aa1-835fd574d6aa admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:09.915325 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f1a40-3d27-4768-9b01-f1d7ec40d6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:09.916185 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f1a40-3d27-4768-9b01-f1d7ec40d6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:09.916466 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f1a40-3d27-4768-9b01-f1d7ec40d6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:09.916743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f1a40-3d27-4768-9b01-f1d7ec40d6d5 admin admin] PATH_INFO: `/v3/users/27902c80add140c1b83fe26316b6f73e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:09.917917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f1a40-3d27-4768-9b01-f1d7ec40d6d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27902c80add140c1b83fe26316b6f73e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:09.919872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f1a40-3d27-4768-9b01-f1d7ec40d6d5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:10.004773 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 911/1827] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:09 2022] PUT /identity/v3/projects/f1046fd13d5343c491f716f6f5a71c93/users/fad6c305398f44299afa208a2aa36711/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:10.005674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19943184-4c57-4ee4-a114-95f1debccf80 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:10.064288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19943184-4c57-4ee4-a114-95f1debccf80 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:10.069670 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d047826b-5106-474f-8d1f-d1aa753bbc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:10.070367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047826b-5106-474f-8d1f-d1aa753bbc22 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:10.070600 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047826b-5106-474f-8d1f-d1aa753bbc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:10.070874 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047826b-5106-474f-8d1f-d1aa753bbc22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:10.071626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d047826b-5106-474f-8d1f-d1aa753bbc22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:10.073208 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d047826b-5106-474f-8d1f-d1aa753bbc22 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:10.083481 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 912/1828] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:10.084817 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99968771-f14c-4bf4-ab04-6699974bbec1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:10.085103 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99968771-f14c-4bf4-ab04-6699974bbec1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:10.085381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99968771-f14c-4bf4-ab04-6699974bbec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:10.113376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-99968771-f14c-4bf4-ab04-6699974bbec1 None None] MFA Rules not processed for user `fad6c305398f44299afa208a2aa36711`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:10.114649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-99968771-f14c-4bf4-ab04-6699974bbec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:10.132851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-005f1a40-3d27-4768-9b01-f1d7ec40d6d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27902c80add140c1b83fe26316b6f73e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:10.167318 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-005f1a40-3d27-4768-9b01-f1d7ec40d6d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27902c80add140c1b83fe26316b6f73e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:10.168006 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-005f1a40-3d27-4768-9b01-f1d7ec40d6d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27902c80add140c1b83fe26316b6f73e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:10.180332 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 913/1829] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:10.181042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99968771-f14c-4bf4-ab04-6699974bbec1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:10.184490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-005f1a40-3d27-4768-9b01-f1d7ec40d6d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27902c80add140c1b83fe26316b6f73e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:10.185197 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-005f1a40-3d27-4768-9b01-f1d7ec40d6d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27902c80add140c1b83fe26316b6f73e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:10.197938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99968771-f14c-4bf4-ab04-6699974bbec1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:10.208083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce32f6-bd06-4f76-bd4e-8827d4e6aafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:10.208991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce32f6-bd06-4f76-bd4e-8827d4e6aafa admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:10.209273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce32f6-bd06-4f76-bd4e-8827d4e6aafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:10.209934 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce32f6-bd06-4f76-bd4e-8827d4e6aafa admin admin] PATH_INFO: `/v3/projects/601602c13b8d427187453003a154bca7/users/ec0f30ec26d74be1b881ae049f70a01f/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:10.228884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce32f6-bd06-4f76-bd4e-8827d4e6aafa admin admin] RBAC: Authorizing `identity:create_grant(project_id=601602c13b8d427187453003a154bca7, user_id=ec0f30ec26d74be1b881ae049f70a01f, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:10.231972 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce32f6-bd06-4f76-bd4e-8827d4e6aafa admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:10.268960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 917/1830] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:09 2022] DELETE /identity/v3/users/27902c80add140c1b83fe26316b6f73e => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:10.272575 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f1a40-3d27-4768-9b01-f1d7ec40d6d5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:10.272575 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-005f1a40-3d27-4768-9b01-f1d7ec40d6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:10.331223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 914/1831] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:10 2022] PUT /identity/v3/projects/601602c13b8d427187453003a154bca7/users/ec0f30ec26d74be1b881ae049f70a01f/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:10.344360 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9983fb9-3e86-48af-b396-7a73f0f0d6d1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:10.344799 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9983fb9-3e86-48af-b396-7a73f0f0d6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:10.345466 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9983fb9-3e86-48af-b396-7a73f0f0d6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:10.371712 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f1a40-3d27-4768-9b01-f1d7ec40d6d5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:10.388537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c9983fb9-3e86-48af-b396-7a73f0f0d6d1 None None] MFA Rules not processed for user `ec0f30ec26d74be1b881ae049f70a01f`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:10.391497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c9983fb9-3e86-48af-b396-7a73f0f0d6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:10.508582 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:10.511206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 915/1832] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:10.521495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9983fb9-3e86-48af-b396-7a73f0f0d6d1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:10.524303 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c9983fb9-3e86-48af-b396-7a73f0f0d6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:10.538674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cc9b4-30be-436c-bb3f-bf6117576c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:10.539416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cc9b4-30be-436c-bb3f-bf6117576c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:10.540199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cc9b4-30be-436c-bb3f-bf6117576c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:10.540399 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cc9b4-30be-436c-bb3f-bf6117576c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:10.541116 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cc9b4-30be-436c-bb3f-bf6117576c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:10.548347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cc9b4-30be-436c-bb3f-bf6117576c46 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:10.558540 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9983fb9-3e86-48af-b396-7a73f0f0d6d1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:10.565121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9592ea1-5c20-4c4c-9f0d-a7551b7211e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:10.567228 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9592ea1-5c20-4c4c-9f0d-a7551b7211e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:10.570156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9592ea1-5c20-4c4c-9f0d-a7551b7211e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:10.570156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9592ea1-5c20-4c4c-9f0d-a7551b7211e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:10.570156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9592ea1-5c20-4c4c-9f0d-a7551b7211e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:10.573823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9592ea1-5c20-4c4c-9f0d-a7551b7211e4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:10.676166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 918/1833] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:10 2022] POST /identity/v3/projects => generated 371 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:10.680582 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cc9b4-30be-436c-bb3f-bf6117576c46 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:10.707089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 916/1834] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:10 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:10.710457 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9592ea1-5c20-4c4c-9f0d-a7551b7211e4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:10.768476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cc9b4-30be-436c-bb3f-bf6117576c46 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:10.822559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9592ea1-5c20-4c4c-9f0d-a7551b7211e4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:10.822559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b372d-607d-43b7-8a61-440d8bde0994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:10.824652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b372d-607d-43b7-8a61-440d8bde0994 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:10.824985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b372d-607d-43b7-8a61-440d8bde0994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:10.825268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b372d-607d-43b7-8a61-440d8bde0994 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:10.826022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b372d-607d-43b7-8a61-440d8bde0994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:10.828558 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b372d-607d-43b7-8a61-440d8bde0994 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:10.883695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46583c41-7f51-4c3b-985e-242d94aacab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:10.886111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46583c41-7f51-4c3b-985e-242d94aacab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:10.886552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46583c41-7f51-4c3b-985e-242d94aacab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:10.887965 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46583c41-7f51-4c3b-985e-242d94aacab6 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:10.889692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46583c41-7f51-4c3b-985e-242d94aacab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:10.893506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46583c41-7f51-4c3b-985e-242d94aacab6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:11.856831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 917/1835] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:10 2022] POST /identity/v3/users => generated 299 bytes in 1147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:11.860682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b372d-607d-43b7-8a61-440d8bde0994 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:11.870219 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b372d-607d-43b7-8a61-440d8bde0994 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:11.874984 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288a53c-6299-4851-87aa-ca9c31b0828f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:11.875678 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288a53c-6299-4851-87aa-ca9c31b0828f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:11.875967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288a53c-6299-4851-87aa-ca9c31b0828f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:11.876574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288a53c-6299-4851-87aa-ca9c31b0828f admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:11.876954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288a53c-6299-4851-87aa-ca9c31b0828f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:11.878616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288a53c-6299-4851-87aa-ca9c31b0828f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:11.888464 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 918/1836] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:19:11.892615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288a53c-6299-4851-87aa-ca9c31b0828f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:11.904908 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:11.908665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 919/1837] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:10 2022] POST /identity/v3/users => generated 292 bytes in 1229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:11.910369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46583c41-7f51-4c3b-985e-242d94aacab6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:11.913315 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-46583c41-7f51-4c3b-985e-242d94aacab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:11.913972 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288a53c-6299-4851-87aa-ca9c31b0828f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:11.919878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab1385-adcb-4a5c-80d3-78dcc5fc92b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:11.921030 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab1385-adcb-4a5c-80d3-78dcc5fc92b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:11.921318 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab1385-adcb-4a5c-80d3-78dcc5fc92b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:11.921595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab1385-adcb-4a5c-80d3-78dcc5fc92b0 admin admin] PATH_INFO: `/v3/projects/4046f31f68c84c1db9eb2ffe6bd2043e/users/24eb0e0579944204aa2a79a98d1cc9f9/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:11.933446 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46583c41-7f51-4c3b-985e-242d94aacab6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:11.939227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458e32a-c458-4592-9a89-35e44d0dc2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:11.940091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458e32a-c458-4592-9a89-35e44d0dc2e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:11.940376 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458e32a-c458-4592-9a89-35e44d0dc2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:11.940697 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458e32a-c458-4592-9a89-35e44d0dc2e5 admin admin] PATH_INFO: `/v3/projects/6a28bb1df9134de9a179b6b8021b8d5d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:11.942014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458e32a-c458-4592-9a89-35e44d0dc2e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a28bb1df9134de9a179b6b8021b8d5d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:11.943958 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458e32a-c458-4592-9a89-35e44d0dc2e5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:11.952931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab1385-adcb-4a5c-80d3-78dcc5fc92b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4046f31f68c84c1db9eb2ffe6bd2043e, user_id=24eb0e0579944204aa2a79a98d1cc9f9, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:11.955704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab1385-adcb-4a5c-80d3-78dcc5fc92b0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:12.094827 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 919/1838] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:11 2022] PUT /identity/v3/projects/4046f31f68c84c1db9eb2ffe6bd2043e/users/24eb0e0579944204aa2a79a98d1cc9f9/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:12.096399 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab1385-adcb-4a5c-80d3-78dcc5fc92b0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:12.118190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a458e32a-c458-4592-9a89-35e44d0dc2e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a28bb1df9134de9a179b6b8021b8d5d', 'request_id': 'req-a458e32a-c458-4592-9a89-35e44d0dc2e5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:12.126142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a458e32a-c458-4592-9a89-35e44d0dc2e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a28bb1df9134de9a179b6b8021b8d5d', 'request_id': 'req-a458e32a-c458-4592-9a89-35e44d0dc2e5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:12.210970 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab1385-adcb-4a5c-80d3-78dcc5fc92b0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:12.218688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a458e32a-c458-4592-9a89-35e44d0dc2e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a28bb1df9134de9a179b6b8021b8d5d', 'request_id': 'req-a458e32a-c458-4592-9a89-35e44d0dc2e5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:12.243508 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a458e32a-c458-4592-9a89-35e44d0dc2e5 admin admin] The token cache is being invalidate because project 6a28bb1df9134de9a179b6b8021b8d5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:12.244791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a458e32a-c458-4592-9a89-35e44d0dc2e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:12.247607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a458e32a-c458-4592-9a89-35e44d0dc2e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:12.257691 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 920/1839] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:11 2022] DELETE /identity/v3/projects/6a28bb1df9134de9a179b6b8021b8d5d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:12.257691 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458e32a-c458-4592-9a89-35e44d0dc2e5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:12.262323 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a458e32a-c458-4592-9a89-35e44d0dc2e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:12.307429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0ccc7-2ac7-4b8e-a6a3-822c5e1950af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:12.312512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0ccc7-2ac7-4b8e-a6a3-822c5e1950af admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:12.312512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0ccc7-2ac7-4b8e-a6a3-822c5e1950af admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:12.312512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0ccc7-2ac7-4b8e-a6a3-822c5e1950af admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:12.312512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0ccc7-2ac7-4b8e-a6a3-822c5e1950af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:12.313230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0ccc7-2ac7-4b8e-a6a3-822c5e1950af admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:12.333314 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 920/1840] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:12.335901 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1a215-c522-4753-afcc-7463b90aa81d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:12.336201 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1a215-c522-4753-afcc-7463b90aa81d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:12.336484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1a215-c522-4753-afcc-7463b90aa81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:12.360224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458e32a-c458-4592-9a89-35e44d0dc2e5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:12.364135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f9f1a215-c522-4753-afcc-7463b90aa81d None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:12.364317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f9f1a215-c522-4753-afcc-7463b90aa81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:12.371837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd92aa2-60c2-4073-a3e1-f814b08bd755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:12.372610 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd92aa2-60c2-4073-a3e1-f814b08bd755 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:12.372954 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd92aa2-60c2-4073-a3e1-f814b08bd755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:12.373548 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd92aa2-60c2-4073-a3e1-f814b08bd755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:12.374505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd92aa2-60c2-4073-a3e1-f814b08bd755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:12.375763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 921/1841] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:12.376595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1a215-c522-4753-afcc-7463b90aa81d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:12.380228 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd92aa2-60c2-4073-a3e1-f814b08bd755 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:12.384139 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f9f1a215-c522-4753-afcc-7463b90aa81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:12.399450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1a215-c522-4753-afcc-7463b90aa81d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:12.402336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 921/1842] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:12.404297 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cce13-b68e-47e8-a021-45f1fb5a0a92 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:12.404703 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cce13-b68e-47e8-a021-45f1fb5a0a92 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:12.405314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cce13-b68e-47e8-a021-45f1fb5a0a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:12.419432 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc3ef7-5b1e-463f-8933-daa9d07d9c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:12.419432 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc3ef7-5b1e-463f-8933-daa9d07d9c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:12.419432 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc3ef7-5b1e-463f-8933-daa9d07d9c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:12.419432 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc3ef7-5b1e-463f-8933-daa9d07d9c77 admin admin] PATH_INFO: `/v3/projects/670c34bbd1bd40979e2e3c331a1790ae/users/513d0e20e17e4ffb9a393b72cfea67f0/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:12.438681 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc3ef7-5b1e-463f-8933-daa9d07d9c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=670c34bbd1bd40979e2e3c331a1790ae, user_id=513d0e20e17e4ffb9a393b72cfea67f0, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:12.443064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc3ef7-5b1e-463f-8933-daa9d07d9c77 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:12.452615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6e3cce13-b68e-47e8-a021-45f1fb5a0a92 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:12.454220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6e3cce13-b68e-47e8-a021-45f1fb5a0a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:12.464500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 922/1843] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:12.465480 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3cce13-b68e-47e8-a021-45f1fb5a0a92 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:12.482394 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3cce13-b68e-47e8-a021-45f1fb5a0a92 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:12.487426 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb20e3-bdd6-4155-ba13-02a2e1858a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:12.488503 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb20e3-bdd6-4155-ba13-02a2e1858a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:12.488858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb20e3-bdd6-4155-ba13-02a2e1858a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:12.489256 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb20e3-bdd6-4155-ba13-02a2e1858a2a admin admin] PATH_INFO: `/v3/projects/4046f31f68c84c1db9eb2ffe6bd2043e/users/24eb0e0579944204aa2a79a98d1cc9f9/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:12.491456 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb20e3-bdd6-4155-ba13-02a2e1858a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4046f31f68c84c1db9eb2ffe6bd2043e, user_id=24eb0e0579944204aa2a79a98d1cc9f9, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:12.495418 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb20e3-bdd6-4155-ba13-02a2e1858a2a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:12.506024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 923/1844] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:12 2022] PUT /identity/v3/projects/4046f31f68c84c1db9eb2ffe6bd2043e/users/24eb0e0579944204aa2a79a98d1cc9f9/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:12.510949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb20e3-bdd6-4155-ba13-02a2e1858a2a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:12.538047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 922/1845] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:12 2022] PUT /identity/v3/projects/670c34bbd1bd40979e2e3c331a1790ae/users/513d0e20e17e4ffb9a393b72cfea67f0/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:12.539681 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c3615-4fd4-4483-9578-4e85e26d3c24 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:12.540144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c3615-4fd4-4483-9578-4e85e26d3c24 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:12.545584 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c3615-4fd4-4483-9578-4e85e26d3c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:12.568684 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a72c3615-4fd4-4483-9578-4e85e26d3c24 None None] MFA Rules not processed for user `24eb0e0579944204aa2a79a98d1cc9f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:12.570133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a72c3615-4fd4-4483-9578-4e85e26d3c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:12.597272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb20e3-bdd6-4155-ba13-02a2e1858a2a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:12.683846 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 923/1846] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:12.683846 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c3615-4fd4-4483-9578-4e85e26d3c24 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:12.729447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaaa3fd-3604-4cc2-8a12-aeb642c727e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:12.730307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaa3fd-3604-4cc2-8a12-aeb642c727e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:12.730903 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaa3fd-3604-4cc2-8a12-aeb642c727e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:12.731376 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaa3fd-3604-4cc2-8a12-aeb642c727e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:12.732521 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaaa3fd-3604-4cc2-8a12-aeb642c727e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:19:12.733086 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaaa3fd-3604-4cc2-8a12-aeb642c727e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:12.735456 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaaa3fd-3604-4cc2-8a12-aeb642c727e0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:12.753034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 924/1847] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:19:12.755206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaaa3fd-3604-4cc2-8a12-aeb642c727e0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:12.757136 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-efaaa3fd-3604-4cc2-8a12-aeb642c727e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:12.781623 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c3615-4fd4-4483-9578-4e85e26d3c24 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:12.787571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a365d0c-8681-40d4-bd21-af879fb64a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:12.789936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a365d0c-8681-40d4-bd21-af879fb64a10 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:12.789936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a365d0c-8681-40d4-bd21-af879fb64a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:12.789936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a365d0c-8681-40d4-bd21-af879fb64a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:12.789936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a365d0c-8681-40d4-bd21-af879fb64a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:12.791380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a365d0c-8681-40d4-bd21-af879fb64a10 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:12.802036 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 924/1848] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:12.802957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a365d0c-8681-40d4-bd21-af879fb64a10 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:12.818446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a365d0c-8681-40d4-bd21-af879fb64a10 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:12.820940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde6379-e1dd-406e-ac3b-7feadff6fb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:12.821613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde6379-e1dd-406e-ac3b-7feadff6fb6c admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:12.822135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde6379-e1dd-406e-ac3b-7feadff6fb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:12.822240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde6379-e1dd-406e-ac3b-7feadff6fb6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:12.822789 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaaa3fd-3604-4cc2-8a12-aeb642c727e0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:12.822887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde6379-e1dd-406e-ac3b-7feadff6fb6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:12.824915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde6379-e1dd-406e-ac3b-7feadff6fb6c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:12.827256 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a88dc-4ad8-4516-87ff-73795a5f8466 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:12.827988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a88dc-4ad8-4516-87ff-73795a5f8466 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:12.828344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a88dc-4ad8-4516-87ff-73795a5f8466 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:12.828695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a88dc-4ad8-4516-87ff-73795a5f8466 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:12.841328 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a88dc-4ad8-4516-87ff-73795a5f8466 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:12.842871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a88dc-4ad8-4516-87ff-73795a5f8466 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:12.854778 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 925/1849] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:12.855629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a88dc-4ad8-4516-87ff-73795a5f8466 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:12.868945 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a88dc-4ad8-4516-87ff-73795a5f8466 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:12.876296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c56bfa-e64e-49bb-bdef-6e2cb7a3c01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:12.876296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c56bfa-e64e-49bb-bdef-6e2cb7a3c01f admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:12.876296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c56bfa-e64e-49bb-bdef-6e2cb7a3c01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:12.876296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c56bfa-e64e-49bb-bdef-6e2cb7a3c01f admin admin] PATH_INFO: `/v3/projects/670c34bbd1bd40979e2e3c331a1790ae/users/513d0e20e17e4ffb9a393b72cfea67f0/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:12.877942 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c56bfa-e64e-49bb-bdef-6e2cb7a3c01f admin admin] RBAC: Authorizing `identity:create_grant(project_id=670c34bbd1bd40979e2e3c331a1790ae, user_id=513d0e20e17e4ffb9a393b72cfea67f0, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:12.882417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c56bfa-e64e-49bb-bdef-6e2cb7a3c01f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:12.901506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 926/1850] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:12 2022] PUT /identity/v3/projects/670c34bbd1bd40979e2e3c331a1790ae/users/513d0e20e17e4ffb9a393b72cfea67f0/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:12.918334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74fe2a-ebff-4d6c-9b28-b80f057414a6 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:12.918939 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74fe2a-ebff-4d6c-9b28-b80f057414a6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:12.918939 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74fe2a-ebff-4d6c-9b28-b80f057414a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:12.933732 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 925/1851] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:12 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:12.942230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde6379-e1dd-406e-ac3b-7feadff6fb6c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:12.955104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4c74fe2a-ebff-4d6c-9b28-b80f057414a6 None None] MFA Rules not processed for user `513d0e20e17e4ffb9a393b72cfea67f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:12.956458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4c74fe2a-ebff-4d6c-9b28-b80f057414a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:13.004343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:13.012883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde6379-e1dd-406e-ac3b-7feadff6fb6c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:13.017516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3420615-1102-4687-afec-92ba59f98c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:13.018185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3420615-1102-4687-afec-92ba59f98c63 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:13.018468 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3420615-1102-4687-afec-92ba59f98c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:13.018771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3420615-1102-4687-afec-92ba59f98c63 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:13.019560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3420615-1102-4687-afec-92ba59f98c63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:13.021533 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3420615-1102-4687-afec-92ba59f98c63 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:13.038292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 927/1852] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:13.039116 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74fe2a-ebff-4d6c-9b28-b80f057414a6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:13.048872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74fe2a-ebff-4d6c-9b28-b80f057414a6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:13.054219 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a41ba4-46c7-4ec7-b7fc-8cb8d5e9a5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:13.055079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a41ba4-46c7-4ec7-b7fc-8cb8d5e9a5a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:13.055391 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a41ba4-46c7-4ec7-b7fc-8cb8d5e9a5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:13.055665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a41ba4-46c7-4ec7-b7fc-8cb8d5e9a5a2 admin admin] PATH_INFO: `/v3/users/ec0f30ec26d74be1b881ae049f70a01f` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:13.056857 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a41ba4-46c7-4ec7-b7fc-8cb8d5e9a5a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec0f30ec26d74be1b881ae049f70a01f)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:13.058751 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a41ba4-46c7-4ec7-b7fc-8cb8d5e9a5a2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:13.119433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 926/1853] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:12 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:13.121675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3420615-1102-4687-afec-92ba59f98c63 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:13.122035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f3420615-1102-4687-afec-92ba59f98c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:13.191051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3420615-1102-4687-afec-92ba59f98c63 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:13.196526 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f19e4a-1869-4c20-917c-86bfaec7e70b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:13.197198 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f19e4a-1869-4c20-917c-86bfaec7e70b service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:13.197483 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f19e4a-1869-4c20-917c-86bfaec7e70b service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:13.197764 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f19e4a-1869-4c20-917c-86bfaec7e70b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:13.208792 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f19e4a-1869-4c20-917c-86bfaec7e70b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:13.210245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f19e4a-1869-4c20-917c-86bfaec7e70b service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:13.222435 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 927/1854] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:13 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:13.224418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f19e4a-1869-4c20-917c-86bfaec7e70b service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:13.234578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f19e4a-1869-4c20-917c-86bfaec7e70b service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:13.241401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acea61-babf-4120-b995-0d4e445dc459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:13.243403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acea61-babf-4120-b995-0d4e445dc459 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:13.247081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acea61-babf-4120-b995-0d4e445dc459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:13.247081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acea61-babf-4120-b995-0d4e445dc459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:13.247081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acea61-babf-4120-b995-0d4e445dc459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:13.247367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acea61-babf-4120-b995-0d4e445dc459 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:13.262635 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 928/1855] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:13.266506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acea61-babf-4120-b995-0d4e445dc459 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:13.277228 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acea61-babf-4120-b995-0d4e445dc459 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:13.282194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af4374-15f0-4964-a6c6-ee18a729b465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:13.282870 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af4374-15f0-4964-a6c6-ee18a729b465 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:13.283157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af4374-15f0-4964-a6c6-ee18a729b465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:13.283464 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af4374-15f0-4964-a6c6-ee18a729b465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:13.284197 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af4374-15f0-4964-a6c6-ee18a729b465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:13.286403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af4374-15f0-4964-a6c6-ee18a729b465 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:13.297993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 929/1856] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:19:13.303093 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af4374-15f0-4964-a6c6-ee18a729b465 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:13.321804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af4374-15f0-4964-a6c6-ee18a729b465 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:13.332738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05a41ba4-46c7-4ec7-b7fc-8cb8d5e9a5a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec0f30ec26d74be1b881ae049f70a01f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:13.337170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2d0b4-7e95-4282-ae8a-9f618d59ef2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:13.338085 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2d0b4-7e95-4282-ae8a-9f618d59ef2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:13.338369 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2d0b4-7e95-4282-ae8a-9f618d59ef2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:13.338648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2d0b4-7e95-4282-ae8a-9f618d59ef2d admin admin] PATH_INFO: `/v3/projects/aceba606649d4f828748383a1b72a3d2/users/8488d3b1578d42fb9cc38ade0fa0b892/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:13.360938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05a41ba4-46c7-4ec7-b7fc-8cb8d5e9a5a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec0f30ec26d74be1b881ae049f70a01f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:13.363522 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05a41ba4-46c7-4ec7-b7fc-8cb8d5e9a5a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec0f30ec26d74be1b881ae049f70a01f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:13.380766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2d0b4-7e95-4282-ae8a-9f618d59ef2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aceba606649d4f828748383a1b72a3d2, user_id=8488d3b1578d42fb9cc38ade0fa0b892, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:13.380864 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05a41ba4-46c7-4ec7-b7fc-8cb8d5e9a5a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec0f30ec26d74be1b881ae049f70a01f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:13.380864 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05a41ba4-46c7-4ec7-b7fc-8cb8d5e9a5a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec0f30ec26d74be1b881ae049f70a01f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:13.382955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2d0b4-7e95-4282-ae8a-9f618d59ef2d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:13.410973 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:13.551061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 928/1857] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:13 2022] DELETE /identity/v3/users/ec0f30ec26d74be1b881ae049f70a01f => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:13.564689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 930/1858] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:13 2022] PUT /identity/v3/projects/aceba606649d4f828748383a1b72a3d2/users/8488d3b1578d42fb9cc38ade0fa0b892/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:13.566378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b269cfb-7ebd-467e-93fa-0af1897997a0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:13.566689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b269cfb-7ebd-467e-93fa-0af1897997a0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:13.570722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b269cfb-7ebd-467e-93fa-0af1897997a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:13.570849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a41ba4-46c7-4ec7-b7fc-8cb8d5e9a5a2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:13.571452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-05a41ba4-46c7-4ec7-b7fc-8cb8d5e9a5a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:13.610858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6b269cfb-7ebd-467e-93fa-0af1897997a0 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:13.616507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6b269cfb-7ebd-467e-93fa-0af1897997a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:13.670297 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a41ba4-46c7-4ec7-b7fc-8cb8d5e9a5a2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:13.674876 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818e17c-b2bf-487f-8e3c-3126e4b8a835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:13.676022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818e17c-b2bf-487f-8e3c-3126e4b8a835 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:13.676387 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818e17c-b2bf-487f-8e3c-3126e4b8a835 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:13.676743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818e17c-b2bf-487f-8e3c-3126e4b8a835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:13.678870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6818e17c-b2bf-487f-8e3c-3126e4b8a835 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:13.709196 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 931/1859] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:13.710533 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add22b1-feaa-4fad-a0de-537101e72504 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:13.710889 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add22b1-feaa-4fad-a0de-537101e72504 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:13.711240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add22b1-feaa-4fad-a0de-537101e72504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:13.757499 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2add22b1-feaa-4fad-a0de-537101e72504 None None] MFA Rules not processed for user `8488d3b1578d42fb9cc38ade0fa0b892`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:13.766778 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2add22b1-feaa-4fad-a0de-537101e72504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:13.796463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818e17c-b2bf-487f-8e3c-3126e4b8a835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:13.800073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818e17c-b2bf-487f-8e3c-3126e4b8a835 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:13.812854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 929/1860] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:13.813716 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818e17c-b2bf-487f-8e3c-3126e4b8a835 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:13.815354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6818e17c-b2bf-487f-8e3c-3126e4b8a835 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:13.859705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 932/1861] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:13.878859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add22b1-feaa-4fad-a0de-537101e72504 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:13.881281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2add22b1-feaa-4fad-a0de-537101e72504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:13.894873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add22b1-feaa-4fad-a0de-537101e72504 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:13.898726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fbf8de-abb2-4583-a660-5f3ac7e3d0fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:13.899412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbf8de-abb2-4583-a660-5f3ac7e3d0fc service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:13.904450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbf8de-abb2-4583-a660-5f3ac7e3d0fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:13.904450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbf8de-abb2-4583-a660-5f3ac7e3d0fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:13.908632 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818e17c-b2bf-487f-8e3c-3126e4b8a835 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:13.911745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fbf8de-abb2-4583-a660-5f3ac7e3d0fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:13.913469 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fbf8de-abb2-4583-a660-5f3ac7e3d0fc service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:13.914642 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc99eb-badc-4ec5-a892-a217a9bfa59c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:13.915420 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc99eb-badc-4ec5-a892-a217a9bfa59c service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:13.915838 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc99eb-badc-4ec5-a892-a217a9bfa59c service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:13.916693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc99eb-badc-4ec5-a892-a217a9bfa59c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:13.924176 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 933/1862] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:13.928108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc99eb-badc-4ec5-a892-a217a9bfa59c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:13.929645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc99eb-badc-4ec5-a892-a217a9bfa59c service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:13.940806 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 930/1863] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:13.944428 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fbf8de-abb2-4583-a660-5f3ac7e3d0fc service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:13.946116 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-82fbf8de-abb2-4583-a660-5f3ac7e3d0fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:13.964563 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fbf8de-abb2-4583-a660-5f3ac7e3d0fc service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:13.986850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf00ff47-37f4-4a50-82c0-3c28ac79a24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:13.988992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00ff47-37f4-4a50-82c0-3c28ac79a24a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:13.990628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00ff47-37f4-4a50-82c0-3c28ac79a24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:13.990628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00ff47-37f4-4a50-82c0-3c28ac79a24a admin admin] PATH_INFO: `/v3/users/8488d3b1578d42fb9cc38ade0fa0b892` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:13.991089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf00ff47-37f4-4a50-82c0-3c28ac79a24a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8488d3b1578d42fb9cc38ade0fa0b892)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:13.993540 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf00ff47-37f4-4a50-82c0-3c28ac79a24a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:14.211827 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bf00ff47-37f4-4a50-82c0-3c28ac79a24a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8488d3b1578d42fb9cc38ade0fa0b892'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:14.227720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bf00ff47-37f4-4a50-82c0-3c28ac79a24a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8488d3b1578d42fb9cc38ade0fa0b892'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:14.296816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:14.323726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bf00ff47-37f4-4a50-82c0-3c28ac79a24a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8488d3b1578d42fb9cc38ade0fa0b892'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:14.323726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bf00ff47-37f4-4a50-82c0-3c28ac79a24a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8488d3b1578d42fb9cc38ade0fa0b892'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:14.338729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bf00ff47-37f4-4a50-82c0-3c28ac79a24a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8488d3b1578d42fb9cc38ade0fa0b892'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:14.350570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 934/1864] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:13 2022] DELETE /identity/v3/users/8488d3b1578d42fb9cc38ade0fa0b892 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:14.355192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856086c-ad09-4eda-8676-4a963738d0a4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:14.355479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856086c-ad09-4eda-8676-4a963738d0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:14.355760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856086c-ad09-4eda-8676-4a963738d0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:14.383308 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0856086c-ad09-4eda-8676-4a963738d0a4 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:14.387329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0856086c-ad09-4eda-8676-4a963738d0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:14.506139 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af1c7c-107e-47f6-a0cc-dc5c987bb3f0 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:14.506592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af1c7c-107e-47f6-a0cc-dc5c987bb3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:14.507537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af1c7c-107e-47f6-a0cc-dc5c987bb3f0 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:14.509810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 935/1865] 10.222.0.67 () {54 vars in 921 bytes} [Fri Jun 24 15:19:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:19:14.520479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec96765-886c-4bf3-8195-ed01c11763e3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:14.521192 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec96765-886c-4bf3-8195-ed01c11763e3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:14.521192 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec96765-886c-4bf3-8195-ed01c11763e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:14.558800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4ec96765-886c-4bf3-8195-ed01c11763e3 None None] MFA Rules not processed for user `51de59caa3144a1cbf12b20b15e2134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:14.561329 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4ec96765-886c-4bf3-8195-ed01c11763e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:14.573106 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:14.581946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 931/1866] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:14.587637 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856086c-ad09-4eda-8676-4a963738d0a4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:14.589496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0856086c-ad09-4eda-8676-4a963738d0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:14.707403 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856086c-ad09-4eda-8676-4a963738d0a4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:14.715132 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 936/1867] 10.222.0.67 () {58 vars in 1018 bytes} [Fri Jun 24 15:19:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:14.717991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136704cb-c225-4745-82d2-b59a4c98aef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:14.720569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136704cb-c225-4745-82d2-b59a4c98aef2 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:14.721096 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136704cb-c225-4745-82d2-b59a4c98aef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:14.721667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136704cb-c225-4745-82d2-b59a4c98aef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:14.729292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec96765-886c-4bf3-8195-ed01c11763e3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:14.731202 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4ec96765-886c-4bf3-8195-ed01c11763e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:14.758620 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136704cb-c225-4745-82d2-b59a4c98aef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:14.762432 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136704cb-c225-4745-82d2-b59a4c98aef2 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:14.766349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec96765-886c-4bf3-8195-ed01c11763e3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:14.780725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9af7cd-14d4-453c-a50a-2cb96a771075 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:14.781648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9af7cd-14d4-453c-a50a-2cb96a771075 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:14.781929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9af7cd-14d4-453c-a50a-2cb96a771075 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:14.782204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9af7cd-14d4-453c-a50a-2cb96a771075 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:14.789743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 932/1868] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:19:14.801081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9af7cd-14d4-453c-a50a-2cb96a771075 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:14.803467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9af7cd-14d4-453c-a50a-2cb96a771075 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:14.816571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 937/1869] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:19:15.067038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136704cb-c225-4745-82d2-b59a4c98aef2 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:15.069289 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-136704cb-c225-4745-82d2-b59a4c98aef2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:15.092927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136704cb-c225-4745-82d2-b59a4c98aef2 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:15.105374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e8e6b-3938-4a57-99d5-88c4c9748904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:15.106369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e8e6b-3938-4a57-99d5-88c4c9748904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:15.106654 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e8e6b-3938-4a57-99d5-88c4c9748904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:15.106926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e8e6b-3938-4a57-99d5-88c4c9748904 admin admin] PATH_INFO: `/v3/projects/601602c13b8d427187453003a154bca7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:15.108303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e8e6b-3938-4a57-99d5-88c4c9748904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=601602c13b8d427187453003a154bca7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:15.110369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e8e6b-3938-4a57-99d5-88c4c9748904 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:15.299798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5e1e8e6b-3938-4a57-99d5-88c4c9748904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '601602c13b8d427187453003a154bca7', 'request_id': 'req-5e1e8e6b-3938-4a57-99d5-88c4c9748904'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:15.308754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5e1e8e6b-3938-4a57-99d5-88c4c9748904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '601602c13b8d427187453003a154bca7', 'request_id': 'req-5e1e8e6b-3938-4a57-99d5-88c4c9748904'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:15.386855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5e1e8e6b-3938-4a57-99d5-88c4c9748904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '601602c13b8d427187453003a154bca7', 'request_id': 'req-5e1e8e6b-3938-4a57-99d5-88c4c9748904'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:15.400704 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5e1e8e6b-3938-4a57-99d5-88c4c9748904 admin admin] The token cache is being invalidate because project 601602c13b8d427187453003a154bca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:15.400998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5e1e8e6b-3938-4a57-99d5-88c4c9748904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:15.401826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5e1e8e6b-3938-4a57-99d5-88c4c9748904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:15.403419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 933/1870] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:15 2022] DELETE /identity/v3/projects/601602c13b8d427187453003a154bca7 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:15.407661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9af7cd-14d4-453c-a50a-2cb96a771075 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:15.418380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9af7cd-14d4-453c-a50a-2cb96a771075 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:15.475092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ec609-44b5-41d4-9e4c-37d863c1cc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:15.475963 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ec609-44b5-41d4-9e4c-37d863c1cc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:15.476273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ec609-44b5-41d4-9e4c-37d863c1cc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:15.476608 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ec609-44b5-41d4-9e4c-37d863c1cc7a admin admin] PATH_INFO: `/v3/users/24eb0e0579944204aa2a79a98d1cc9f9` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:15.477985 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ec609-44b5-41d4-9e4c-37d863c1cc7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=24eb0e0579944204aa2a79a98d1cc9f9)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:15.480045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ec609-44b5-41d4-9e4c-37d863c1cc7a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:15.513325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:15.722799 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d27ec609-44b5-41d4-9e4c-37d863c1cc7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24eb0e0579944204aa2a79a98d1cc9f9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:15.743162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d27ec609-44b5-41d4-9e4c-37d863c1cc7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24eb0e0579944204aa2a79a98d1cc9f9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:15.838919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d27ec609-44b5-41d4-9e4c-37d863c1cc7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24eb0e0579944204aa2a79a98d1cc9f9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:15.839205 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:15.839892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d27ec609-44b5-41d4-9e4c-37d863c1cc7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24eb0e0579944204aa2a79a98d1cc9f9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:15.848633 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d27ec609-44b5-41d4-9e4c-37d863c1cc7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24eb0e0579944204aa2a79a98d1cc9f9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:15.855683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 938/1871] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:15 2022] DELETE /identity/v3/users/24eb0e0579944204aa2a79a98d1cc9f9 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:15.920409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e8e6b-3938-4a57-99d5-88c4c9748904 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:15.921921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5e1e8e6b-3938-4a57-99d5-88c4c9748904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:15.937079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e8e6b-3938-4a57-99d5-88c4c9748904 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:16.029625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d2e79-116e-4feb-88c0-45469e208426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:16.030506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d2e79-116e-4feb-88c0-45469e208426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:16.031304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d2e79-116e-4feb-88c0-45469e208426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:16.031400 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d2e79-116e-4feb-88c0-45469e208426 admin admin] PATH_INFO: `/v3/projects/aceba606649d4f828748383a1b72a3d2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:16.032651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d2e79-116e-4feb-88c0-45469e208426 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aceba606649d4f828748383a1b72a3d2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:16.035555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d2e79-116e-4feb-88c0-45469e208426 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:16.177793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-821d2e79-116e-4feb-88c0-45469e208426 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aceba606649d4f828748383a1b72a3d2', 'request_id': 'req-821d2e79-116e-4feb-88c0-45469e208426'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:16.201494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-821d2e79-116e-4feb-88c0-45469e208426 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aceba606649d4f828748383a1b72a3d2', 'request_id': 'req-821d2e79-116e-4feb-88c0-45469e208426'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:16.308896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-821d2e79-116e-4feb-88c0-45469e208426 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aceba606649d4f828748383a1b72a3d2', 'request_id': 'req-821d2e79-116e-4feb-88c0-45469e208426'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:16.345817 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-821d2e79-116e-4feb-88c0-45469e208426 admin admin] The token cache is being invalidate because project aceba606649d4f828748383a1b72a3d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:16.348370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-821d2e79-116e-4feb-88c0-45469e208426 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:16.350653 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-821d2e79-116e-4feb-88c0-45469e208426 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:16.356541 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 934/1872] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:15 2022] DELETE /identity/v3/projects/aceba606649d4f828748383a1b72a3d2 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:16.361861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6184386-32f0-40b9-b4e8-a62405f355a3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:16.361861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6184386-32f0-40b9-b4e8-a62405f355a3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:16.362263 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6184386-32f0-40b9-b4e8-a62405f355a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:16.404190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a6184386-32f0-40b9-b4e8-a62405f355a3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:16.405407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a6184386-32f0-40b9-b4e8-a62405f355a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:16.470589 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d2e79-116e-4feb-88c0-45469e208426 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:16.472938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-821d2e79-116e-4feb-88c0-45469e208426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:16.514908 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 939/1873] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:16.523525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9b8b8-a7d8-41c4-9dbd-3650d351fb94 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:16.523938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9b8b8-a7d8-41c4-9dbd-3650d351fb94 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:16.524313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9b8b8-a7d8-41c4-9dbd-3650d351fb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:16.575546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f4e9b8b8-a7d8-41c4-9dbd-3650d351fb94 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:16.576771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f4e9b8b8-a7d8-41c4-9dbd-3650d351fb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:16.595069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 940/1874] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:16.601216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9b8b8-a7d8-41c4-9dbd-3650d351fb94 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:16.601475 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d2e79-116e-4feb-88c0-45469e208426 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:16.607241 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23da7ee-190e-4d45-8fef-ed386a4d4da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:16.608320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23da7ee-190e-4d45-8fef-ed386a4d4da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:16.608661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23da7ee-190e-4d45-8fef-ed386a4d4da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:16.609003 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23da7ee-190e-4d45-8fef-ed386a4d4da6 admin admin] PATH_INFO: `/v3/projects/4046f31f68c84c1db9eb2ffe6bd2043e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:16.610294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23da7ee-190e-4d45-8fef-ed386a4d4da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4046f31f68c84c1db9eb2ffe6bd2043e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:16.610762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9b8b8-a7d8-41c4-9dbd-3650d351fb94 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:16.613144 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23da7ee-190e-4d45-8fef-ed386a4d4da6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:16.616658 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e04ae1-3c6a-4246-9a85-871f3a04a1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:16.618593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e04ae1-3c6a-4246-9a85-871f3a04a1ca admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:16.618593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e04ae1-3c6a-4246-9a85-871f3a04a1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:16.618593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e04ae1-3c6a-4246-9a85-871f3a04a1ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:16.619221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e04ae1-3c6a-4246-9a85-871f3a04a1ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:19:16.619453 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e04ae1-3c6a-4246-9a85-871f3a04a1ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:16.621200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e04ae1-3c6a-4246-9a85-871f3a04a1ca admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:16.635349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 941/1875] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:19:16.638424 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e04ae1-3c6a-4246-9a85-871f3a04a1ca admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:16.649949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e04ae1-3c6a-4246-9a85-871f3a04a1ca admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:16.654751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55b923-d012-4a13-886f-b78b77c102cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:16.655456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55b923-d012-4a13-886f-b78b77c102cd admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:16.655747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55b923-d012-4a13-886f-b78b77c102cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:16.656282 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55b923-d012-4a13-886f-b78b77c102cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:16.656982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55b923-d012-4a13-886f-b78b77c102cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:16.658960 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55b923-d012-4a13-886f-b78b77c102cd admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:16.725192 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:16.745713 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c23da7ee-190e-4d45-8fef-ed386a4d4da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4046f31f68c84c1db9eb2ffe6bd2043e', 'request_id': 'req-c23da7ee-190e-4d45-8fef-ed386a4d4da6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:16.752601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c23da7ee-190e-4d45-8fef-ed386a4d4da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4046f31f68c84c1db9eb2ffe6bd2043e', 'request_id': 'req-c23da7ee-190e-4d45-8fef-ed386a4d4da6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:16.788755 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 942/1876] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:16 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:16.790127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55b923-d012-4a13-886f-b78b77c102cd admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:16.873182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55b923-d012-4a13-886f-b78b77c102cd admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:16.878799 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7590a-7ead-4cf2-96d7-7e5e33b8547c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:16.879648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7590a-7ead-4cf2-96d7-7e5e33b8547c admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:16.879935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7590a-7ead-4cf2-96d7-7e5e33b8547c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:16.880217 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7590a-7ead-4cf2-96d7-7e5e33b8547c admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:16.880915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7590a-7ead-4cf2-96d7-7e5e33b8547c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:16.882878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7590a-7ead-4cf2-96d7-7e5e33b8547c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:16.906324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c23da7ee-190e-4d45-8fef-ed386a4d4da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4046f31f68c84c1db9eb2ffe6bd2043e', 'request_id': 'req-c23da7ee-190e-4d45-8fef-ed386a4d4da6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:16.918374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c23da7ee-190e-4d45-8fef-ed386a4d4da6 admin admin] The token cache is being invalidate because project 4046f31f68c84c1db9eb2ffe6bd2043e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:16.918665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c23da7ee-190e-4d45-8fef-ed386a4d4da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:16.919346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c23da7ee-190e-4d45-8fef-ed386a4d4da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:16.920899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 935/1877] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:16 2022] DELETE /identity/v3/projects/4046f31f68c84c1db9eb2ffe6bd2043e => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:16.926355 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1f867-901b-4fb4-aadf-51946155c913 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:16.926646 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1f867-901b-4fb4-aadf-51946155c913 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:16.926932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1f867-901b-4fb4-aadf-51946155c913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:16.946764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-bca1f867-901b-4fb4-aadf-51946155c913 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:16.948103 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bca1f867-901b-4fb4-aadf-51946155c913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:16.956706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 936/1878] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:16.967856 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913ffef-6a85-482e-8c2d-a2efc59e2fb4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:16.968149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913ffef-6a85-482e-8c2d-a2efc59e2fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:16.968429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913ffef-6a85-482e-8c2d-a2efc59e2fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:16.986728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0913ffef-6a85-482e-8c2d-a2efc59e2fb4 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:16.990445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 943/1879] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:16 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:16.990556 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0913ffef-6a85-482e-8c2d-a2efc59e2fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:16.993121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7590a-7ead-4cf2-96d7-7e5e33b8547c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:16.994691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4dc7590a-7ead-4cf2-96d7-7e5e33b8547c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:16.996634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 937/1880] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:17.000537 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913ffef-6a85-482e-8c2d-a2efc59e2fb4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:17.007357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7590a-7ead-4cf2-96d7-7e5e33b8547c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:17.009583 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913ffef-6a85-482e-8c2d-a2efc59e2fb4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:17.013202 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c64e94-ca93-4060-b4ed-1aff1cd6d73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:17.013918 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c64e94-ca93-4060-b4ed-1aff1cd6d73a admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:17.014229 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c64e94-ca93-4060-b4ed-1aff1cd6d73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:17.014509 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c64e94-ca93-4060-b4ed-1aff1cd6d73a admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:17.015341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c64e94-ca93-4060-b4ed-1aff1cd6d73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:17.015459 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e0b0c-8df5-4487-bdd6-4549b64a3bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:17.019064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e0b0c-8df5-4487-bdd6-4549b64a3bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:17.019064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e0b0c-8df5-4487-bdd6-4549b64a3bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:17.019064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e0b0c-8df5-4487-bdd6-4549b64a3bf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:17.019337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c64e94-ca93-4060-b4ed-1aff1cd6d73a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:17.019463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e0b0c-8df5-4487-bdd6-4549b64a3bf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:19:17.019463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e0b0c-8df5-4487-bdd6-4549b64a3bf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:17.019703 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e0b0c-8df5-4487-bdd6-4549b64a3bf9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:17.028520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 944/1881] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:19:17.032843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c64e94-ca93-4060-b4ed-1aff1cd6d73a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:17.035131 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 938/1882] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:19:17.036201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e0b0c-8df5-4487-bdd6-4549b64a3bf9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:17.043803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c64e94-ca93-4060-b4ed-1aff1cd6d73a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:17.046134 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e0b0c-8df5-4487-bdd6-4549b64a3bf9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:17.048547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6565d54-0f60-4a0a-87df-3692badbb0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:17.049230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6565d54-0f60-4a0a-87df-3692badbb0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:17.049520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6565d54-0f60-4a0a-87df-3692badbb0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:17.049931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6565d54-0f60-4a0a-87df-3692badbb0b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:17.051170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6565d54-0f60-4a0a-87df-3692badbb0b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:17.051729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eef240-9659-445f-9423-845e429e538c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:17.052379 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eef240-9659-445f-9423-845e429e538c admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:17.052659 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eef240-9659-445f-9423-845e429e538c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:17.052848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6565d54-0f60-4a0a-87df-3692badbb0b2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:17.052947 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eef240-9659-445f-9423-845e429e538c admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:17.053611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eef240-9659-445f-9423-845e429e538c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:17.055536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eef240-9659-445f-9423-845e429e538c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:17.063926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 945/1883] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:19:17.068381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6565d54-0f60-4a0a-87df-3692badbb0b2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:17.078317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6565d54-0f60-4a0a-87df-3692badbb0b2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:17.083995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe6f47-6026-4dc7-9d72-7f28159baac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:17.084872 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe6f47-6026-4dc7-9d72-7f28159baac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:17.085156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe6f47-6026-4dc7-9d72-7f28159baac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:17.085453 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe6f47-6026-4dc7-9d72-7f28159baac3 admin admin] PATH_INFO: `/v3/projects/5a3257f43c7c4c19a3a74209b64aa27f/users/e1442c910c774b308533174792e31a84/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:17.101574 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe6f47-6026-4dc7-9d72-7f28159baac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a3257f43c7c4c19a3a74209b64aa27f, user_id=e1442c910c774b308533174792e31a84, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:17.104034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe6f47-6026-4dc7-9d72-7f28159baac3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:17.154125 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 939/1884] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:17 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:17.158281 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eef240-9659-445f-9423-845e429e538c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:17.182741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 946/1885] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:17 2022] PUT /identity/v3/projects/5a3257f43c7c4c19a3a74209b64aa27f/users/e1442c910c774b308533174792e31a84/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:17.193342 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5221fba-a30d-4353-9748-93a1b9166fa8 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:17.193642 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5221fba-a30d-4353-9748-93a1b9166fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:17.193922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5221fba-a30d-4353-9748-93a1b9166fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:17.220691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b5221fba-a30d-4353-9748-93a1b9166fa8 None None] MFA Rules not processed for user `e1442c910c774b308533174792e31a84`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:17.222536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b5221fba-a30d-4353-9748-93a1b9166fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:17.224554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eef240-9659-445f-9423-845e429e538c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:17.280453 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3d4ef-d8a2-4f72-ab3f-6246832d854b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:17.281309 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3d4ef-d8a2-4f72-ab3f-6246832d854b admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:17.281594 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3d4ef-d8a2-4f72-ab3f-6246832d854b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:17.281870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3d4ef-d8a2-4f72-ab3f-6246832d854b admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:17.282587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3d4ef-d8a2-4f72-ab3f-6246832d854b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:17.285559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3d4ef-d8a2-4f72-ab3f-6246832d854b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:17.289235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 947/1886] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:17.300148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5221fba-a30d-4353-9748-93a1b9166fa8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:17.301586 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b5221fba-a30d-4353-9748-93a1b9166fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:17.364867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5221fba-a30d-4353-9748-93a1b9166fa8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:17.368558 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bf897-8b5b-4052-a247-91680c62ab63 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1971c464293f426f97d6cb199e48193b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:17.369207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bf897-8b5b-4052-a247-91680c62ab63 service glance] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:17.369495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bf897-8b5b-4052-a247-91680c62ab63 service glance] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:17.369773 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bf897-8b5b-4052-a247-91680c62ab63 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:17.378171 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bf897-8b5b-4052-a247-91680c62ab63 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:17.379915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bf897-8b5b-4052-a247-91680c62ab63 service glance] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:17.388969 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 948/1887] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:19:17.433818 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 940/1888] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:17 2022] POST /identity/v3/users => generated 292 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:17.438014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bf897-8b5b-4052-a247-91680c62ab63 service glance] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:17.449349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bf897-8b5b-4052-a247-91680c62ab63 service glance] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:17.458520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4299c-28ea-48d0-8e97-c57fe7c140ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:17.459486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4299c-28ea-48d0-8e97-c57fe7c140ab admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:17.459907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4299c-28ea-48d0-8e97-c57fe7c140ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:17.460258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4299c-28ea-48d0-8e97-c57fe7c140ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:17.461236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4299c-28ea-48d0-8e97-c57fe7c140ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:17.463027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4299c-28ea-48d0-8e97-c57fe7c140ab admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:17.490676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 949/1889] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:17.491666 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3d4ef-d8a2-4f72-ab3f-6246832d854b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:17.502279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3d4ef-d8a2-4f72-ab3f-6246832d854b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:17.520311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61ef51-2552-498a-9eda-aca2cb51135c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:17.521016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61ef51-2552-498a-9eda-aca2cb51135c admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:17.521311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61ef51-2552-498a-9eda-aca2cb51135c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:17.521597 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61ef51-2552-498a-9eda-aca2cb51135c admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:17.522339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea61ef51-2552-498a-9eda-aca2cb51135c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:17.524201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea61ef51-2552-498a-9eda-aca2cb51135c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:17.538925 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 941/1890] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:17.543489 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4299c-28ea-48d0-8e97-c57fe7c140ab admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:17.560949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4299c-28ea-48d0-8e97-c57fe7c140ab admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:17.571486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de7ef1-ec1e-4976-95a1-2a291e7e24e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:17.572798 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de7ef1-ec1e-4976-95a1-2a291e7e24e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:17.573224 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de7ef1-ec1e-4976-95a1-2a291e7e24e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:17.573613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de7ef1-ec1e-4976-95a1-2a291e7e24e3 admin admin] PATH_INFO: `/v3/projects/55258f2339d14a68ad4b0b19e1922e7c/users/fa3028d6a5f94bf0be210da76b9acc51/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:17.591972 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de7ef1-ec1e-4976-95a1-2a291e7e24e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55258f2339d14a68ad4b0b19e1922e7c, user_id=fa3028d6a5f94bf0be210da76b9acc51, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:17.594722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de7ef1-ec1e-4976-95a1-2a291e7e24e3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:17.686087 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 950/1891] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:17 2022] PUT /identity/v3/projects/55258f2339d14a68ad4b0b19e1922e7c/users/fa3028d6a5f94bf0be210da76b9acc51/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:17.700013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861db63-d304-4086-99a1-8dd868726d12 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:17.700308 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861db63-d304-4086-99a1-8dd868726d12 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:17.700593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861db63-d304-4086-99a1-8dd868726d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:17.732587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9861db63-d304-4086-99a1-8dd868726d12 None None] MFA Rules not processed for user `fa3028d6a5f94bf0be210da76b9acc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:17.737017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9861db63-d304-4086-99a1-8dd868726d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:17.737187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de7ef1-ec1e-4976-95a1-2a291e7e24e3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:17.738942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-04de7ef1-ec1e-4976-95a1-2a291e7e24e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:17.807724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de7ef1-ec1e-4976-95a1-2a291e7e24e3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:17.810688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 942/1892] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:17.812690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9427331-639a-4ac2-803b-b81452d49509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:17.814959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9427331-639a-4ac2-803b-b81452d49509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:17.814959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9427331-639a-4ac2-803b-b81452d49509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:17.814959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9427331-639a-4ac2-803b-b81452d49509 admin admin] PATH_INFO: `/v3/users/43ab5fc92e2c4dadaa137e6f2aa42a69` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:17.815254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861db63-d304-4086-99a1-8dd868726d12 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:17.815400 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9427331-639a-4ac2-803b-b81452d49509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43ab5fc92e2c4dadaa137e6f2aa42a69)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:17.818161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9427331-639a-4ac2-803b-b81452d49509 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:17.826547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861db63-d304-4086-99a1-8dd868726d12 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:17.837436 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7db39-50fa-44b1-a9f6-e5b7272495c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:17.843593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7db39-50fa-44b1-a9f6-e5b7272495c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:17.851454 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7db39-50fa-44b1-a9f6-e5b7272495c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:17.852007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7db39-50fa-44b1-a9f6-e5b7272495c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:17.853030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7db39-50fa-44b1-a9f6-e5b7272495c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:17.857242 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7db39-50fa-44b1-a9f6-e5b7272495c8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:17.950807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:17.963928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 943/1893] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:17 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:17.966892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7db39-50fa-44b1-a9f6-e5b7272495c8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:18.039719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7db39-50fa-44b1-a9f6-e5b7272495c8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:18.045793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe745951-6c25-4d01-9b70-266a67183d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:18.046721 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe745951-6c25-4d01-9b70-266a67183d82 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:18.047419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe745951-6c25-4d01-9b70-266a67183d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:18.047893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b9427331-639a-4ac2-803b-b81452d49509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43ab5fc92e2c4dadaa137e6f2aa42a69'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:18.048365 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe745951-6c25-4d01-9b70-266a67183d82 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:18.051105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe745951-6c25-4d01-9b70-266a67183d82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:18.053467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe745951-6c25-4d01-9b70-266a67183d82 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:18.073176 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b9427331-639a-4ac2-803b-b81452d49509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43ab5fc92e2c4dadaa137e6f2aa42a69'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:18.135718 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:18.158571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b9427331-639a-4ac2-803b-b81452d49509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43ab5fc92e2c4dadaa137e6f2aa42a69'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:18.159325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b9427331-639a-4ac2-803b-b81452d49509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43ab5fc92e2c4dadaa137e6f2aa42a69'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:18.165329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 944/1894] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:17 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:18.169580 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b9427331-639a-4ac2-803b-b81452d49509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43ab5fc92e2c4dadaa137e6f2aa42a69'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:18.169765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe745951-6c25-4d01-9b70-266a67183d82 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:18.170867 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fe745951-6c25-4d01-9b70-266a67183d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:18.173191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 951/1895] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:17 2022] DELETE /identity/v3/users/43ab5fc92e2c4dadaa137e6f2aa42a69 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:18.178222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16debc2-7926-4511-ab86-5dca01c09712 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:18.178585 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16debc2-7926-4511-ab86-5dca01c09712 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:18.178926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16debc2-7926-4511-ab86-5dca01c09712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:18.183859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe745951-6c25-4d01-9b70-266a67183d82 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:18.198734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c16debc2-7926-4511-ab86-5dca01c09712 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:18.200115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c16debc2-7926-4511-ab86-5dca01c09712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:18.242934 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed86f88-407a-4609-be67-c7f9732598eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:18.243802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86f88-407a-4609-be67-c7f9732598eb admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:18.244405 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86f88-407a-4609-be67-c7f9732598eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:18.244617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86f88-407a-4609-be67-c7f9732598eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:18.245673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed86f88-407a-4609-be67-c7f9732598eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:18.247517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed86f88-407a-4609-be67-c7f9732598eb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:18.258850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 945/1896] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:19:18.263158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed86f88-407a-4609-be67-c7f9732598eb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:18.269295 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 952/1897] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:18.277327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16debc2-7926-4511-ab86-5dca01c09712 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:18.278418 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed86f88-407a-4609-be67-c7f9732598eb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:18.279088 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c16debc2-7926-4511-ab86-5dca01c09712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:18.283643 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e934be-ff7b-434b-8d65-5515a1036d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:18.285253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e934be-ff7b-434b-8d65-5515a1036d19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:18.285927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e934be-ff7b-434b-8d65-5515a1036d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:18.286217 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e934be-ff7b-434b-8d65-5515a1036d19 admin admin] PATH_INFO: `/v3/projects/da35aa256dae43a3828f03bb1e8492bc/users/9110b90eb29a4d78bad72b644b7d1a5f/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:18.304145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e934be-ff7b-434b-8d65-5515a1036d19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da35aa256dae43a3828f03bb1e8492bc, user_id=9110b90eb29a4d78bad72b644b7d1a5f, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:18.308391 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e934be-ff7b-434b-8d65-5515a1036d19 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:18.349033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16debc2-7926-4511-ab86-5dca01c09712 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:18.350932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61076cdc-cf9f-4652-870a-e32cc41fa1d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:18.351832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61076cdc-cf9f-4652-870a-e32cc41fa1d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:18.352194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61076cdc-cf9f-4652-870a-e32cc41fa1d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:18.352548 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61076cdc-cf9f-4652-870a-e32cc41fa1d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:18.366415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61076cdc-cf9f-4652-870a-e32cc41fa1d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:18.368855 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61076cdc-cf9f-4652-870a-e32cc41fa1d1 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:18.395022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 953/1898] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:18.423641 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 946/1899] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:18 2022] PUT /identity/v3/projects/da35aa256dae43a3828f03bb1e8492bc/users/9110b90eb29a4d78bad72b644b7d1a5f/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:18.429081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61076cdc-cf9f-4652-870a-e32cc41fa1d1 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:18.442624 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61076cdc-cf9f-4652-870a-e32cc41fa1d1 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:18.507708 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0d981-6c4a-41ea-b510-c2a0b9898be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:18.508407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0d981-6c4a-41ea-b510-c2a0b9898be2 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:18.508690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0d981-6c4a-41ea-b510-c2a0b9898be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:18.508977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0d981-6c4a-41ea-b510-c2a0b9898be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:18.509686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d0d981-6c4a-41ea-b510-c2a0b9898be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:18.511342 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d0d981-6c4a-41ea-b510-c2a0b9898be2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:18.521246 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 954/1900] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:18.525508 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e934be-ff7b-434b-8d65-5515a1036d19 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:18.537458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e934be-ff7b-434b-8d65-5515a1036d19 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:18.542467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c671d-bb08-4f6a-8bbb-92f5c0732dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:18.543389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c671d-bb08-4f6a-8bbb-92f5c0732dcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:18.543678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c671d-bb08-4f6a-8bbb-92f5c0732dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:18.543961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c671d-bb08-4f6a-8bbb-92f5c0732dcd admin admin] PATH_INFO: `/v3/projects/da35aa256dae43a3828f03bb1e8492bc/users/9110b90eb29a4d78bad72b644b7d1a5f/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:18.545995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c671d-bb08-4f6a-8bbb-92f5c0732dcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=da35aa256dae43a3828f03bb1e8492bc, user_id=9110b90eb29a4d78bad72b644b7d1a5f, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:18.548722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c671d-bb08-4f6a-8bbb-92f5c0732dcd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:18.563447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 947/1901] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:18 2022] PUT /identity/v3/projects/da35aa256dae43a3828f03bb1e8492bc/users/9110b90eb29a4d78bad72b644b7d1a5f/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:18.571438 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd9e86-4875-4b3f-8b9d-cd5ea82e7b8b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:18.571913 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd9e86-4875-4b3f-8b9d-cd5ea82e7b8b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:18.572019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd9e86-4875-4b3f-8b9d-cd5ea82e7b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:18.601888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bedd9e86-4875-4b3f-8b9d-cd5ea82e7b8b None None] MFA Rules not processed for user `9110b90eb29a4d78bad72b644b7d1a5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:18.603468 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bedd9e86-4875-4b3f-8b9d-cd5ea82e7b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:18.694480 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 955/1902] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:18.705849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c671d-bb08-4f6a-8bbb-92f5c0732dcd admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:18.708671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5a9c671d-bb08-4f6a-8bbb-92f5c0732dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:18.769835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c671d-bb08-4f6a-8bbb-92f5c0732dcd admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:18.773695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e630b-4417-433d-8fff-f1265573d5bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:18.774351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e630b-4417-433d-8fff-f1265573d5bc service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:18.775748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e630b-4417-433d-8fff-f1265573d5bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:18.775748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e630b-4417-433d-8fff-f1265573d5bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:18.786116 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e630b-4417-433d-8fff-f1265573d5bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:18.787591 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e630b-4417-433d-8fff-f1265573d5bc service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:18.797956 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 948/1903] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:18.901814 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd9e86-4875-4b3f-8b9d-cd5ea82e7b8b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:18.912531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd9e86-4875-4b3f-8b9d-cd5ea82e7b8b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:18.915506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e630b-4417-433d-8fff-f1265573d5bc service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:18.916918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5d9e630b-4417-433d-8fff-f1265573d5bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:18.917630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6241618-2092-4f6c-a1ed-d48dd566de3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:18.918457 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6241618-2092-4f6c-a1ed-d48dd566de3f service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:18.918775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6241618-2092-4f6c-a1ed-d48dd566de3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:18.919337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6241618-2092-4f6c-a1ed-d48dd566de3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:18.924758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e6241618-2092-4f6c-a1ed-d48dd566de3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:18.984045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6241618-2092-4f6c-a1ed-d48dd566de3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:18.985532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6241618-2092-4f6c-a1ed-d48dd566de3f service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:18.992360 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e630b-4417-433d-8fff-f1265573d5bc service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:18.994232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:18.996582 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 956/1904] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:18.997309 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e97ac-dd13-4b04-afee-a2edd10ac954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:18.998179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e97ac-dd13-4b04-afee-a2edd10ac954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:18.999329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e97ac-dd13-4b04-afee-a2edd10ac954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:18.999329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e97ac-dd13-4b04-afee-a2edd10ac954 admin admin] PATH_INFO: `/v3/projects/35a0890819894fd49a8476c5b013cb19` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:19.000233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e97ac-dd13-4b04-afee-a2edd10ac954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35a0890819894fd49a8476c5b013cb19)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:19.002393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e97ac-dd13-4b04-afee-a2edd10ac954 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:19.094588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:19.095856 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b03e97ac-dd13-4b04-afee-a2edd10ac954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35a0890819894fd49a8476c5b013cb19', 'request_id': 'req-b03e97ac-dd13-4b04-afee-a2edd10ac954'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:19.103377 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b03e97ac-dd13-4b04-afee-a2edd10ac954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35a0890819894fd49a8476c5b013cb19', 'request_id': 'req-b03e97ac-dd13-4b04-afee-a2edd10ac954'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:19.187982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b03e97ac-dd13-4b04-afee-a2edd10ac954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35a0890819894fd49a8476c5b013cb19', 'request_id': 'req-b03e97ac-dd13-4b04-afee-a2edd10ac954'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:19.200282 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b03e97ac-dd13-4b04-afee-a2edd10ac954 admin admin] The token cache is being invalidate because project 35a0890819894fd49a8476c5b013cb19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:19.200575 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b03e97ac-dd13-4b04-afee-a2edd10ac954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:19.201233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b03e97ac-dd13-4b04-afee-a2edd10ac954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:19.203190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 949/1905] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:18 2022] DELETE /identity/v3/projects/35a0890819894fd49a8476c5b013cb19 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:19.207704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6241618-2092-4f6c-a1ed-d48dd566de3f service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:19.218918 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6241618-2092-4f6c-a1ed-d48dd566de3f service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:19.281028 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd7b2e-534c-4c4e-ad9d-c79dda109bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:19.281915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd7b2e-534c-4c4e-ad9d-c79dda109bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:19.282199 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd7b2e-534c-4c4e-ad9d-c79dda109bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:19.282475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd7b2e-534c-4c4e-ad9d-c79dda109bc7 admin admin] PATH_INFO: `/v3/users/4b33f52acd374efbaf7ff7730cc32f3e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:19.283769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd7b2e-534c-4c4e-ad9d-c79dda109bc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b33f52acd374efbaf7ff7730cc32f3e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:19.285761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd7b2e-534c-4c4e-ad9d-c79dda109bc7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:19.504664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0dbd7b2e-534c-4c4e-ad9d-c79dda109bc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b33f52acd374efbaf7ff7730cc32f3e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:19.515986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0dbd7b2e-534c-4c4e-ad9d-c79dda109bc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b33f52acd374efbaf7ff7730cc32f3e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:19.593018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0dbd7b2e-534c-4c4e-ad9d-c79dda109bc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b33f52acd374efbaf7ff7730cc32f3e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:19.593644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0dbd7b2e-534c-4c4e-ad9d-c79dda109bc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b33f52acd374efbaf7ff7730cc32f3e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:19.601064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0dbd7b2e-534c-4c4e-ad9d-c79dda109bc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b33f52acd374efbaf7ff7730cc32f3e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:19.606980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 957/1906] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:19 2022] DELETE /identity/v3/users/4b33f52acd374efbaf7ff7730cc32f3e => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:19.666943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:19.754165 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:20.051839 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e97ac-dd13-4b04-afee-a2edd10ac954 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:20.053373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b03e97ac-dd13-4b04-afee-a2edd10ac954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:20.066920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e97ac-dd13-4b04-afee-a2edd10ac954 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:20.129957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51098d94-fd3b-483c-bd2b-0a397d0ca9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:20.130837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51098d94-fd3b-483c-bd2b-0a397d0ca9ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:20.131121 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51098d94-fd3b-483c-bd2b-0a397d0ca9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:20.131433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51098d94-fd3b-483c-bd2b-0a397d0ca9ca admin admin] PATH_INFO: `/v3/projects/5a9e9230178c457cb0a3d4691c6bf2e9` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:20.132623 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51098d94-fd3b-483c-bd2b-0a397d0ca9ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a9e9230178c457cb0a3d4691c6bf2e9)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:20.134555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51098d94-fd3b-483c-bd2b-0a397d0ca9ca admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:20.302043 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-51098d94-fd3b-483c-bd2b-0a397d0ca9ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a9e9230178c457cb0a3d4691c6bf2e9', 'request_id': 'req-51098d94-fd3b-483c-bd2b-0a397d0ca9ca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:20.308046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-51098d94-fd3b-483c-bd2b-0a397d0ca9ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a9e9230178c457cb0a3d4691c6bf2e9', 'request_id': 'req-51098d94-fd3b-483c-bd2b-0a397d0ca9ca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:20.428111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-51098d94-fd3b-483c-bd2b-0a397d0ca9ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a9e9230178c457cb0a3d4691c6bf2e9', 'request_id': 'req-51098d94-fd3b-483c-bd2b-0a397d0ca9ca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:20.448327 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-51098d94-fd3b-483c-bd2b-0a397d0ca9ca admin admin] The token cache is being invalidate because project 5a9e9230178c457cb0a3d4691c6bf2e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:20.448753 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-51098d94-fd3b-483c-bd2b-0a397d0ca9ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:20.449510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-51098d94-fd3b-483c-bd2b-0a397d0ca9ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:20.451794 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 950/1907] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:20 2022] DELETE /identity/v3/projects/5a9e9230178c457cb0a3d4691c6bf2e9 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:20.456857 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:20.458855 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed635bb5-d859-4e61-bf18-e0e8a6cba92b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:20.459514 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed635bb5-d859-4e61-bf18-e0e8a6cba92b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:20.459954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed635bb5-d859-4e61-bf18-e0e8a6cba92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:20.505342 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ed635bb5-d859-4e61-bf18-e0e8a6cba92b None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:20.506754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ed635bb5-d859-4e61-bf18-e0e8a6cba92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:20.573528 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:20.597964 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 958/1908] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:20.599046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f14ec-ff36-42ac-8f11-2643594b4ea3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:20.599558 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f14ec-ff36-42ac-8f11-2643594b4ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:20.599852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f14ec-ff36-42ac-8f11-2643594b4ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:20.626386 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ec5f14ec-ff36-42ac-8f11-2643594b4ea3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:20.629583 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ec5f14ec-ff36-42ac-8f11-2643594b4ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:20.638534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 951/1909] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:20.644655 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed635bb5-d859-4e61-bf18-e0e8a6cba92b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:20.657798 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed635bb5-d859-4e61-bf18-e0e8a6cba92b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:20.663142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba847ed-f582-490d-ae01-aa08dcdfb076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:20.664205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba847ed-f582-490d-ae01-aa08dcdfb076 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:20.664492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba847ed-f582-490d-ae01-aa08dcdfb076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:20.664770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba847ed-f582-490d-ae01-aa08dcdfb076 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:20.665514 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba847ed-f582-490d-ae01-aa08dcdfb076 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:19:20.665823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba847ed-f582-490d-ae01-aa08dcdfb076 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:20.667633 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba847ed-f582-490d-ae01-aa08dcdfb076 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:20.684610 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 959/1910] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:19:20.689296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f14ec-ff36-42ac-8f11-2643594b4ea3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:20.699996 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f14ec-ff36-42ac-8f11-2643594b4ea3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:20.705050 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de80b1-caf3-424d-a8fc-d0625c4b69d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:20.705736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de80b1-caf3-424d-a8fc-d0625c4b69d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:20.706649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de80b1-caf3-424d-a8fc-d0625c4b69d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:20.706940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de80b1-caf3-424d-a8fc-d0625c4b69d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:20.707673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de80b1-caf3-424d-a8fc-d0625c4b69d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:20.709593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de80b1-caf3-424d-a8fc-d0625c4b69d4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:20.834399 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 952/1911] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:20 2022] POST /identity/v3/projects => generated 363 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:20.835630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba847ed-f582-490d-ae01-aa08dcdfb076 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:20.848298 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba847ed-f582-490d-ae01-aa08dcdfb076 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:20.913604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acef4a-408f-4576-b959-0392fa084da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:20.914435 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acef4a-408f-4576-b959-0392fa084da4 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:20.914726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acef4a-408f-4576-b959-0392fa084da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:20.915010 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acef4a-408f-4576-b959-0392fa084da4 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:20.915751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6acef4a-408f-4576-b959-0392fa084da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:20.917725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6acef4a-408f-4576-b959-0392fa084da4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:20.989562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 960/1912] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:20 2022] POST /identity/v3/users => generated 282 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:20.993956 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de80b1-caf3-424d-a8fc-d0625c4b69d4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:21.006820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de80b1-caf3-424d-a8fc-d0625c4b69d4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:21.008941 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:21.016786 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124faed-7cdd-4a82-8f8f-ec1e14794329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:21.017670 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124faed-7cdd-4a82-8f8f-ec1e14794329 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:21.018728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124faed-7cdd-4a82-8f8f-ec1e14794329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:21.019339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124faed-7cdd-4a82-8f8f-ec1e14794329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:21.021506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e124faed-7cdd-4a82-8f8f-ec1e14794329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:21.024184 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e124faed-7cdd-4a82-8f8f-ec1e14794329 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:21.038828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 953/1913] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:21.042529 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acef4a-408f-4576-b959-0392fa084da4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:21.055022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acef4a-408f-4576-b959-0392fa084da4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:21.060965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8942b89-7844-4365-a104-150506660f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:21.062993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8942b89-7844-4365-a104-150506660f3d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:21.062993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8942b89-7844-4365-a104-150506660f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:21.062993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8942b89-7844-4365-a104-150506660f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:21.063371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8942b89-7844-4365-a104-150506660f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:21.065027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8942b89-7844-4365-a104-150506660f3d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:21.077894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 961/1914] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:21.083053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124faed-7cdd-4a82-8f8f-ec1e14794329 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:21.085665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:21.095042 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124faed-7cdd-4a82-8f8f-ec1e14794329 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:21.103990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2792f9-83f0-473e-a621-995cd189b0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:21.104930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2792f9-83f0-473e-a621-995cd189b0a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:21.105677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2792f9-83f0-473e-a621-995cd189b0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:21.105962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2792f9-83f0-473e-a621-995cd189b0a0 admin admin] PATH_INFO: `/v3/projects/442f8f3a3dd24125b2b8092fa6722f2f/users/cee8c9029f2e46c0833e99cfe2b94a4f/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:21.126577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2792f9-83f0-473e-a621-995cd189b0a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=442f8f3a3dd24125b2b8092fa6722f2f, user_id=cee8c9029f2e46c0833e99cfe2b94a4f, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:21.130301 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2792f9-83f0-473e-a621-995cd189b0a0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:21.233137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 954/1915] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:21 2022] PUT /identity/v3/projects/442f8f3a3dd24125b2b8092fa6722f2f/users/cee8c9029f2e46c0833e99cfe2b94a4f/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:21.245161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95640b8-d5af-4293-a1b5-8687392c47b9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:21.245531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95640b8-d5af-4293-a1b5-8687392c47b9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:21.245813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95640b8-d5af-4293-a1b5-8687392c47b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:21.272959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f95640b8-d5af-4293-a1b5-8687392c47b9 None None] MFA Rules not processed for user `cee8c9029f2e46c0833e99cfe2b94a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:21.274171 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f95640b8-d5af-4293-a1b5-8687392c47b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:21.342126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 962/1916] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:21 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:21.347109 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2792f9-83f0-473e-a621-995cd189b0a0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:21.408403 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2792f9-83f0-473e-a621-995cd189b0a0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:21.413652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33db08-f32a-4c99-8ccb-4733041ec2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:21.414318 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33db08-f32a-4c99-8ccb-4733041ec2ca admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:21.414604 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33db08-f32a-4c99-8ccb-4733041ec2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:21.414884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33db08-f32a-4c99-8ccb-4733041ec2ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:21.415617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33db08-f32a-4c99-8ccb-4733041ec2ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:21.417526 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33db08-f32a-4c99-8ccb-4733041ec2ca admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:21.520523 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 955/1917] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:21 2022] POST /identity/v3/projects => generated 363 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:21.524500 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95640b8-d5af-4293-a1b5-8687392c47b9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:21.536874 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95640b8-d5af-4293-a1b5-8687392c47b9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:21.602015 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c8c2e-cbe8-4fea-8fc8-622d93f574d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:21.602679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c8c2e-cbe8-4fea-8fc8-622d93f574d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:21.602969 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c8c2e-cbe8-4fea-8fc8-622d93f574d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:21.603285 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c8c2e-cbe8-4fea-8fc8-622d93f574d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:21.603977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c8c2e-cbe8-4fea-8fc8-622d93f574d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:21.605944 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c8c2e-cbe8-4fea-8fc8-622d93f574d2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:21.693267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 963/1918] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:21 2022] POST /identity/v3/users => generated 288 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:21.697388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33db08-f32a-4c99-8ccb-4733041ec2ca admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:21.706368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:21.713499 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33db08-f32a-4c99-8ccb-4733041ec2ca admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:21.718548 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16458b-c7eb-47cd-9579-dd7ade936378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:21.719264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16458b-c7eb-47cd-9579-dd7ade936378 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:21.719557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16458b-c7eb-47cd-9579-dd7ade936378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:21.720029 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16458b-c7eb-47cd-9579-dd7ade936378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:21.720831 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16458b-c7eb-47cd-9579-dd7ade936378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:21.723526 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16458b-c7eb-47cd-9579-dd7ade936378 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:21.735427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 956/1919] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:21.739677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c8c2e-cbe8-4fea-8fc8-622d93f574d2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:21.750431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c8c2e-cbe8-4fea-8fc8-622d93f574d2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:21.754148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:21.759027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8dc1d-dda9-4378-9fd7-e80a4bc3fcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:21.760235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8dc1d-dda9-4378-9fd7-e80a4bc3fcdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:21.760527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8dc1d-dda9-4378-9fd7-e80a4bc3fcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:21.760809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8dc1d-dda9-4378-9fd7-e80a4bc3fcdc admin admin] PATH_INFO: `/v3/projects/73b345b050cb4ac7bf2f7754cda711f2/users/a946baab8c0240f0b111fe2b8064a6bf/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:21.781296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8dc1d-dda9-4378-9fd7-e80a4bc3fcdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=73b345b050cb4ac7bf2f7754cda711f2, user_id=a946baab8c0240f0b111fe2b8064a6bf, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:21.787267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8dc1d-dda9-4378-9fd7-e80a4bc3fcdc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:21.890629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 964/1920] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:21 2022] PUT /identity/v3/projects/73b345b050cb4ac7bf2f7754cda711f2/users/a946baab8c0240f0b111fe2b8064a6bf/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:21.903534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16458b-c7eb-47cd-9579-dd7ade936378 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:21.915572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16458b-c7eb-47cd-9579-dd7ade936378 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:21.984440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb660d-aa71-48e9-abbf-fb62aeacd255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:21.985118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb660d-aa71-48e9-abbf-fb62aeacd255 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:21.985412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb660d-aa71-48e9-abbf-fb62aeacd255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:21.985691 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb660d-aa71-48e9-abbf-fb62aeacd255 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:21.986419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb660d-aa71-48e9-abbf-fb62aeacd255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:21.988361 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb660d-aa71-48e9-abbf-fb62aeacd255 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:21.998482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 957/1921] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:19:22.002160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8dc1d-dda9-4378-9fd7-e80a4bc3fcdc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:22.012902 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8dc1d-dda9-4378-9fd7-e80a4bc3fcdc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:22.017490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf44bb-266a-4322-ab50-9521b93107aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:22.019013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf44bb-266a-4322-ab50-9521b93107aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:22.019322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf44bb-266a-4322-ab50-9521b93107aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:22.019688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf44bb-266a-4322-ab50-9521b93107aa admin admin] PATH_INFO: `/v3/projects/73b345b050cb4ac7bf2f7754cda711f2/users/a946baab8c0240f0b111fe2b8064a6bf/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:22.021775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf44bb-266a-4322-ab50-9521b93107aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=73b345b050cb4ac7bf2f7754cda711f2, user_id=a946baab8c0240f0b111fe2b8064a6bf, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:22.024436 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf44bb-266a-4322-ab50-9521b93107aa admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:22.035461 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 965/1922] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:22 2022] PUT /identity/v3/projects/73b345b050cb4ac7bf2f7754cda711f2/users/a946baab8c0240f0b111fe2b8064a6bf/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:22.046569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d33aff-7206-4cbe-b396-a0f665359314 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:22.046854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d33aff-7206-4cbe-b396-a0f665359314 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:22.047134 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d33aff-7206-4cbe-b396-a0f665359314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:22.074721 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-26d33aff-7206-4cbe-b396-a0f665359314 None None] MFA Rules not processed for user `a946baab8c0240f0b111fe2b8064a6bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:22.075973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-26d33aff-7206-4cbe-b396-a0f665359314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:22.102777 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:22.124935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:22.148867 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 958/1923] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:22 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:22.160669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf44bb-266a-4322-ab50-9521b93107aa admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:22.162431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d7bf44bb-266a-4322-ab50-9521b93107aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:22.193420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:22.223475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf44bb-266a-4322-ab50-9521b93107aa admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:22.227239 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a70bb-5574-42a4-8fae-c9b87e8a565e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:22.227789 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a70bb-5574-42a4-8fae-c9b87e8a565e service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:22.228120 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a70bb-5574-42a4-8fae-c9b87e8a565e service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:22.228510 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a70bb-5574-42a4-8fae-c9b87e8a565e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:22.233622 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:22.237803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a70bb-5574-42a4-8fae-c9b87e8a565e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:22.239239 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a70bb-5574-42a4-8fae-c9b87e8a565e service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:22.249162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 966/1924] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:22 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:22.411912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d33aff-7206-4cbe-b396-a0f665359314 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:22.421615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d33aff-7206-4cbe-b396-a0f665359314 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:22.425770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bb34d-4eb0-4b29-a3eb-2108e7b2237f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:22.426409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bb34d-4eb0-4b29-a3eb-2108e7b2237f service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:22.426688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bb34d-4eb0-4b29-a3eb-2108e7b2237f service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:22.426961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bb34d-4eb0-4b29-a3eb-2108e7b2237f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:22.454844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:22.480406 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:22.483612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bb34d-4eb0-4b29-a3eb-2108e7b2237f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:22.485153 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bb34d-4eb0-4b29-a3eb-2108e7b2237f service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:22.494681 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 959/1925] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:22 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:22.718876 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:26.386691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a70bb-5574-42a4-8fae-c9b87e8a565e service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:26.449166 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a70bb-5574-42a4-8fae-c9b87e8a565e service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:26.453948 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5148d1c-9473-49e3-bc70-1bc5e7878bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:26.454837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5148d1c-9473-49e3-bc70-1bc5e7878bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:26.455121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5148d1c-9473-49e3-bc70-1bc5e7878bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:26.455463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5148d1c-9473-49e3-bc70-1bc5e7878bf2 admin admin] PATH_INFO: `/v3/users/cee8c9029f2e46c0833e99cfe2b94a4f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:26.457283 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5148d1c-9473-49e3-bc70-1bc5e7878bf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cee8c9029f2e46c0833e99cfe2b94a4f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:26.459341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5148d1c-9473-49e3-bc70-1bc5e7878bf2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:26.648761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5148d1c-9473-49e3-bc70-1bc5e7878bf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cee8c9029f2e46c0833e99cfe2b94a4f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:26.667071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5148d1c-9473-49e3-bc70-1bc5e7878bf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cee8c9029f2e46c0833e99cfe2b94a4f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:26.753675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5148d1c-9473-49e3-bc70-1bc5e7878bf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cee8c9029f2e46c0833e99cfe2b94a4f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:26.754537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5148d1c-9473-49e3-bc70-1bc5e7878bf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cee8c9029f2e46c0833e99cfe2b94a4f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:26.762205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5148d1c-9473-49e3-bc70-1bc5e7878bf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cee8c9029f2e46c0833e99cfe2b94a4f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:26.769123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 967/1926] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:26 2022] DELETE /identity/v3/users/cee8c9029f2e46c0833e99cfe2b94a4f => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:26.773779 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277f2ac-41a5-4546-bb97-902bda036824 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:26.774279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277f2ac-41a5-4546-bb97-902bda036824 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:26.774371 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277f2ac-41a5-4546-bb97-902bda036824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:26.793055 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6277f2ac-41a5-4546-bb97-902bda036824 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:26.794275 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6277f2ac-41a5-4546-bb97-902bda036824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:26.854829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 960/1927] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:26.866053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5148d1c-9473-49e3-bc70-1bc5e7878bf2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:26.867518 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c5148d1c-9473-49e3-bc70-1bc5e7878bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:26.926768 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5148d1c-9473-49e3-bc70-1bc5e7878bf2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:26.930410 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c4598-453a-4f9a-b983-e7fc51d17676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:26.931060 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c4598-453a-4f9a-b983-e7fc51d17676 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:26.931381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c4598-453a-4f9a-b983-e7fc51d17676 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:26.931661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c4598-453a-4f9a-b983-e7fc51d17676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:26.937450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:26.941780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c4598-453a-4f9a-b983-e7fc51d17676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:26.943227 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c4598-453a-4f9a-b983-e7fc51d17676 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:26.952713 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 968/1928] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:27.395065 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277f2ac-41a5-4546-bb97-902bda036824 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:27.396634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6277f2ac-41a5-4546-bb97-902bda036824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:27.408753 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277f2ac-41a5-4546-bb97-902bda036824 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:27.414122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:27.414989 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:27.415307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:27.415586 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457 admin admin] PATH_INFO: `/v3/projects/442f8f3a3dd24125b2b8092fa6722f2f` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:27.416765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=442f8f3a3dd24125b2b8092fa6722f2f)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:27.418655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:27.549802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '442f8f3a3dd24125b2b8092fa6722f2f', 'request_id': 'req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:27.556196 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '442f8f3a3dd24125b2b8092fa6722f2f', 'request_id': 'req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:27.614570 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '442f8f3a3dd24125b2b8092fa6722f2f', 'request_id': 'req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:27.626569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457 admin admin] The token cache is being invalidate because project 442f8f3a3dd24125b2b8092fa6722f2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:27.626901 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:27.628218 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:27.629893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 961/1929] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:27 2022] DELETE /identity/v3/projects/442f8f3a3dd24125b2b8092fa6722f2f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:27.633943 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c4598-453a-4f9a-b983-e7fc51d17676 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:27.643697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c4598-453a-4f9a-b983-e7fc51d17676 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:27.700831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f64c4f-6638-4f9d-9381-e4a23a2caaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:27.701710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f64c4f-6638-4f9d-9381-e4a23a2caaa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:27.701995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f64c4f-6638-4f9d-9381-e4a23a2caaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:27.702272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f64c4f-6638-4f9d-9381-e4a23a2caaa9 admin admin] PATH_INFO: `/v3/users/a946baab8c0240f0b111fe2b8064a6bf` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:27.703514 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f64c4f-6638-4f9d-9381-e4a23a2caaa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a946baab8c0240f0b111fe2b8064a6bf)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:27.705416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f64c4f-6638-4f9d-9381-e4a23a2caaa9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:27.868293 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e4f64c4f-6638-4f9d-9381-e4a23a2caaa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a946baab8c0240f0b111fe2b8064a6bf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:27.880734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e4f64c4f-6638-4f9d-9381-e4a23a2caaa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a946baab8c0240f0b111fe2b8064a6bf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:27.934667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e4f64c4f-6638-4f9d-9381-e4a23a2caaa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a946baab8c0240f0b111fe2b8064a6bf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:27.935319 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e4f64c4f-6638-4f9d-9381-e4a23a2caaa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a946baab8c0240f0b111fe2b8064a6bf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:27.943965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e4f64c4f-6638-4f9d-9381-e4a23a2caaa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a946baab8c0240f0b111fe2b8064a6bf'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:27.950065 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 969/1930] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:27 2022] DELETE /identity/v3/users/a946baab8c0240f0b111fe2b8064a6bf => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:28.387694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:28.389299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:28.400921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:28.404987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4013555-ee00-4dbe-b4f5-8c2c0e3a2457 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:28.463268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc4d1d-3692-4d28-aaa9-707701dd63a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:28.464060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc4d1d-3692-4d28-aaa9-707701dd63a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:28.464349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc4d1d-3692-4d28-aaa9-707701dd63a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:28.464649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc4d1d-3692-4d28-aaa9-707701dd63a4 admin admin] PATH_INFO: `/v3/projects/73b345b050cb4ac7bf2f7754cda711f2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:28.465855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc4d1d-3692-4d28-aaa9-707701dd63a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73b345b050cb4ac7bf2f7754cda711f2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:28.467799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc4d1d-3692-4d28-aaa9-707701dd63a4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:28.593624 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ccbc4d1d-3692-4d28-aaa9-707701dd63a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73b345b050cb4ac7bf2f7754cda711f2', 'request_id': 'req-ccbc4d1d-3692-4d28-aaa9-707701dd63a4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:28.600684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ccbc4d1d-3692-4d28-aaa9-707701dd63a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73b345b050cb4ac7bf2f7754cda711f2', 'request_id': 'req-ccbc4d1d-3692-4d28-aaa9-707701dd63a4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:28.666964 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ccbc4d1d-3692-4d28-aaa9-707701dd63a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73b345b050cb4ac7bf2f7754cda711f2', 'request_id': 'req-ccbc4d1d-3692-4d28-aaa9-707701dd63a4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:28.678417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ccbc4d1d-3692-4d28-aaa9-707701dd63a4 admin admin] The token cache is being invalidate because project 73b345b050cb4ac7bf2f7754cda711f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:28.678710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ccbc4d1d-3692-4d28-aaa9-707701dd63a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:28.679374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ccbc4d1d-3692-4d28-aaa9-707701dd63a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:28.680990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 962/1931] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:28 2022] DELETE /identity/v3/projects/73b345b050cb4ac7bf2f7754cda711f2 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:28.685904 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f099b7-bb52-463b-abd7-19d939740015 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:28.686334 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f099b7-bb52-463b-abd7-19d939740015 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:28.686507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f099b7-bb52-463b-abd7-19d939740015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:28.706839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-67f099b7-bb52-463b-abd7-19d939740015 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:28.708125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-67f099b7-bb52-463b-abd7-19d939740015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:28.767170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 970/1932] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:28.775390 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a786df-e678-41bd-b975-d941c53bda51 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:28.775682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a786df-e678-41bd-b975-d941c53bda51 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:28.775966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a786df-e678-41bd-b975-d941c53bda51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:28.793998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b7a786df-e678-41bd-b975-d941c53bda51 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:28.795245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b7a786df-e678-41bd-b975-d941c53bda51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:28.803051 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 963/1933] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:28.807070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f099b7-bb52-463b-abd7-19d939740015 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:28.817215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f099b7-bb52-463b-abd7-19d939740015 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:28.821951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e162385-e0a0-4724-9c9f-9d917cdfe595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:28.822608 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e162385-e0a0-4724-9c9f-9d917cdfe595 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:28.822909 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e162385-e0a0-4724-9c9f-9d917cdfe595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:28.823166 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e162385-e0a0-4724-9c9f-9d917cdfe595 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:28.823915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e162385-e0a0-4724-9c9f-9d917cdfe595 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:19:28.824221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e162385-e0a0-4724-9c9f-9d917cdfe595 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:28.825804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e162385-e0a0-4724-9c9f-9d917cdfe595 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:28.834691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:28.837807 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 971/1934] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:19:28.848846 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a786df-e678-41bd-b975-d941c53bda51 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:28.858093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a786df-e678-41bd-b975-d941c53bda51 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:28.862784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef4231-d2d8-4c7c-87ac-4c2149343b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:28.863467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef4231-d2d8-4c7c-87ac-4c2149343b8a admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:28.863751 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef4231-d2d8-4c7c-87ac-4c2149343b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:28.864026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef4231-d2d8-4c7c-87ac-4c2149343b8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:28.864711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef4231-d2d8-4c7c-87ac-4c2149343b8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:28.866597 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef4231-d2d8-4c7c-87ac-4c2149343b8a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:28.970728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 964/1935] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:28 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:28.974669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e162385-e0a0-4724-9c9f-9d917cdfe595 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:28.984718 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e162385-e0a0-4724-9c9f-9d917cdfe595 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:29.039926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7740c78-ac36-415a-b225-b7e7fbd8b931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:29.040524 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7740c78-ac36-415a-b225-b7e7fbd8b931 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:29.040813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7740c78-ac36-415a-b225-b7e7fbd8b931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:29.041095 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7740c78-ac36-415a-b225-b7e7fbd8b931 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:29.041787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7740c78-ac36-415a-b225-b7e7fbd8b931 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:29.043734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7740c78-ac36-415a-b225-b7e7fbd8b931 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:29.199642 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 972/1936] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:28 2022] POST /identity/v3/users => generated 290 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:29.204632 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef4231-d2d8-4c7c-87ac-4c2149343b8a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:29.215423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef4231-d2d8-4c7c-87ac-4c2149343b8a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:29.220415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639e828-95d2-4e99-9ac7-4d56f623a085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:29.221090 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639e828-95d2-4e99-9ac7-4d56f623a085 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:29.221376 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639e828-95d2-4e99-9ac7-4d56f623a085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:29.221684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639e828-95d2-4e99-9ac7-4d56f623a085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:29.222417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639e828-95d2-4e99-9ac7-4d56f623a085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:29.224603 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639e828-95d2-4e99-9ac7-4d56f623a085 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:29.235063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 965/1937] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:29.239823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7740c78-ac36-415a-b225-b7e7fbd8b931 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:29.249992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7740c78-ac36-415a-b225-b7e7fbd8b931 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:29.255311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01fa8a-c331-4604-b78e-10f052ae2e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:29.255977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01fa8a-c331-4604-b78e-10f052ae2e38 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:29.256290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01fa8a-c331-4604-b78e-10f052ae2e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:29.256570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01fa8a-c331-4604-b78e-10f052ae2e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:29.257294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01fa8a-c331-4604-b78e-10f052ae2e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:29.258892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01fa8a-c331-4604-b78e-10f052ae2e38 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:29.269515 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 973/1938] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:19:29.273961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639e828-95d2-4e99-9ac7-4d56f623a085 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:29.284417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639e828-95d2-4e99-9ac7-4d56f623a085 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:29.289278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b9451-3dfd-4f8f-a576-391385af19ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:29.290224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b9451-3dfd-4f8f-a576-391385af19ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:29.290453 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b9451-3dfd-4f8f-a576-391385af19ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:29.290736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b9451-3dfd-4f8f-a576-391385af19ba admin admin] PATH_INFO: `/v3/projects/86b3ba387f7d4daca0f6ffbbeff58b29/users/3cedcad33cbe403db994702c2da6b7f0/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:29.316317 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b9451-3dfd-4f8f-a576-391385af19ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=86b3ba387f7d4daca0f6ffbbeff58b29, user_id=3cedcad33cbe403db994702c2da6b7f0, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:29.319110 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b9451-3dfd-4f8f-a576-391385af19ba admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:29.415483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 966/1939] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:29 2022] PUT /identity/v3/projects/86b3ba387f7d4daca0f6ffbbeff58b29/users/3cedcad33cbe403db994702c2da6b7f0/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:29.426763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262e7f7-90a8-4af9-82e8-98e5f6a8898f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:29.427066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262e7f7-90a8-4af9-82e8-98e5f6a8898f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:29.427378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262e7f7-90a8-4af9-82e8-98e5f6a8898f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:29.456441 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3262e7f7-90a8-4af9-82e8-98e5f6a8898f None None] MFA Rules not processed for user `3cedcad33cbe403db994702c2da6b7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:29.457670 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3262e7f7-90a8-4af9-82e8-98e5f6a8898f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:29.525657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 974/1940] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:29.530373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b9451-3dfd-4f8f-a576-391385af19ba admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:29.569226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:29.598123 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b9451-3dfd-4f8f-a576-391385af19ba admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:29.602787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c8d19-f105-4eba-bec9-22c5c23c24bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:29.603481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c8d19-f105-4eba-bec9-22c5c23c24bc admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:29.603765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c8d19-f105-4eba-bec9-22c5c23c24bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:29.604040 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c8d19-f105-4eba-bec9-22c5c23c24bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:29.604729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c8d19-f105-4eba-bec9-22c5c23c24bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:29.606618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c8d19-f105-4eba-bec9-22c5c23c24bc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:29.675588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 967/1941] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:29 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:29.679624 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262e7f7-90a8-4af9-82e8-98e5f6a8898f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:29.690895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262e7f7-90a8-4af9-82e8-98e5f6a8898f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:29.749717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558b526-3a32-40e7-8f20-638740fae9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:29.753367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558b526-3a32-40e7-8f20-638740fae9d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:29.753367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558b526-3a32-40e7-8f20-638740fae9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:29.753367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558b526-3a32-40e7-8f20-638740fae9d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:29.753367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558b526-3a32-40e7-8f20-638740fae9d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:29.754285 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558b526-3a32-40e7-8f20-638740fae9d8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:29.844609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:29.850804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 975/1942] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:29 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:29.854831 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c8d19-f105-4eba-bec9-22c5c23c24bc admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:29.866891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c8d19-f105-4eba-bec9-22c5c23c24bc admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:29.871899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bdf59-a298-48ef-905e-7a78fcdf7d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:29.872598 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bdf59-a298-48ef-905e-7a78fcdf7d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:29.872887 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bdf59-a298-48ef-905e-7a78fcdf7d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:29.873175 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bdf59-a298-48ef-905e-7a78fcdf7d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:29.873892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186bdf59-a298-48ef-905e-7a78fcdf7d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:29.875539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186bdf59-a298-48ef-905e-7a78fcdf7d12 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:29.895731 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 968/1943] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:29.903257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558b526-3a32-40e7-8f20-638740fae9d8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:29.916322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558b526-3a32-40e7-8f20-638740fae9d8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:29.923089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba030b-f843-4858-89a4-b789d1427863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:29.925460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba030b-f843-4858-89a4-b789d1427863 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:29.926034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba030b-f843-4858-89a4-b789d1427863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:29.926397 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba030b-f843-4858-89a4-b789d1427863 admin admin] PATH_INFO: `/v3/projects/57d3ab94e91a491ea40f947ffb4346cb/users/e41222b4e8f14939a71fd9eecec64cff/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:29.946475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba030b-f843-4858-89a4-b789d1427863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57d3ab94e91a491ea40f947ffb4346cb, user_id=e41222b4e8f14939a71fd9eecec64cff, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:29.949671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba030b-f843-4858-89a4-b789d1427863 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:30.044935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 976/1944] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:29 2022] PUT /identity/v3/projects/57d3ab94e91a491ea40f947ffb4346cb/users/e41222b4e8f14939a71fd9eecec64cff/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:30.048992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bdf59-a298-48ef-905e-7a78fcdf7d12 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:30.059949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bdf59-a298-48ef-905e-7a78fcdf7d12 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:30.126840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bdef10-883d-4a7b-a163-2d4f0ae46af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:30.128247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bdef10-883d-4a7b-a163-2d4f0ae46af2 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:30.128636 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bdef10-883d-4a7b-a163-2d4f0ae46af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:30.128921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bdef10-883d-4a7b-a163-2d4f0ae46af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:30.129638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bdef10-883d-4a7b-a163-2d4f0ae46af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:30.131397 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bdef10-883d-4a7b-a163-2d4f0ae46af2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:30.141323 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 969/1945] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:30.145711 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba030b-f843-4858-89a4-b789d1427863 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:30.156523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba030b-f843-4858-89a4-b789d1427863 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:30.161171 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eae77b-c847-40ae-9fd4-8e9a4939b451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:30.162048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eae77b-c847-40ae-9fd4-8e9a4939b451 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:30.162381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eae77b-c847-40ae-9fd4-8e9a4939b451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:30.162660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eae77b-c847-40ae-9fd4-8e9a4939b451 admin admin] PATH_INFO: `/v3/projects/57d3ab94e91a491ea40f947ffb4346cb/users/e41222b4e8f14939a71fd9eecec64cff/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:30.164721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eae77b-c847-40ae-9fd4-8e9a4939b451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57d3ab94e91a491ea40f947ffb4346cb, user_id=e41222b4e8f14939a71fd9eecec64cff, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:30.167486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eae77b-c847-40ae-9fd4-8e9a4939b451 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:30.179323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 977/1946] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:30 2022] PUT /identity/v3/projects/57d3ab94e91a491ea40f947ffb4346cb/users/e41222b4e8f14939a71fd9eecec64cff/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:30.191728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2560626-717c-491a-8974-aed05b66fe2c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:30.192019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2560626-717c-491a-8974-aed05b66fe2c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:30.192305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2560626-717c-491a-8974-aed05b66fe2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:30.219549 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b2560626-717c-491a-8974-aed05b66fe2c None None] MFA Rules not processed for user `e41222b4e8f14939a71fd9eecec64cff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:30.219930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eae77b-c847-40ae-9fd4-8e9a4939b451 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:30.220777 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b2560626-717c-491a-8974-aed05b66fe2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:30.222236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-45eae77b-c847-40ae-9fd4-8e9a4939b451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:30.306405 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eae77b-c847-40ae-9fd4-8e9a4939b451 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:30.311245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3dad1-2fc2-4532-9972-1b4af659cae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:30.312144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3dad1-2fc2-4532-9972-1b4af659cae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:30.312430 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3dad1-2fc2-4532-9972-1b4af659cae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:30.312711 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3dad1-2fc2-4532-9972-1b4af659cae8 admin admin] PATH_INFO: `/v3/users/fa3028d6a5f94bf0be210da76b9acc51` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:30.313974 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3dad1-2fc2-4532-9972-1b4af659cae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa3028d6a5f94bf0be210da76b9acc51)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:30.316465 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3dad1-2fc2-4532-9972-1b4af659cae8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:30.325601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 970/1947] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:30.339131 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2560626-717c-491a-8974-aed05b66fe2c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:30.340695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b2560626-717c-491a-8974-aed05b66fe2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:30.396511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:30.430774 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2560626-717c-491a-8974-aed05b66fe2c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:30.434635 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25b178-c4bf-4ccd-930b-34ec8ae1587d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:30.435943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25b178-c4bf-4ccd-930b-34ec8ae1587d service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:30.436250 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25b178-c4bf-4ccd-930b-34ec8ae1587d service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:30.436538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25b178-c4bf-4ccd-930b-34ec8ae1587d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:30.453950 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25b178-c4bf-4ccd-930b-34ec8ae1587d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:30.455674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25b178-c4bf-4ccd-930b-34ec8ae1587d service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:30.475133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 971/1948] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:19:30.526381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fca3dad1-2fc2-4532-9972-1b4af659cae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa3028d6a5f94bf0be210da76b9acc51'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:30.541710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fca3dad1-2fc2-4532-9972-1b4af659cae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa3028d6a5f94bf0be210da76b9acc51'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:30.675339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fca3dad1-2fc2-4532-9972-1b4af659cae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa3028d6a5f94bf0be210da76b9acc51'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:30.675992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fca3dad1-2fc2-4532-9972-1b4af659cae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa3028d6a5f94bf0be210da76b9acc51'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:30.684069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fca3dad1-2fc2-4532-9972-1b4af659cae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa3028d6a5f94bf0be210da76b9acc51'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:30.691470 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 978/1949] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:30 2022] DELETE /identity/v3/users/fa3028d6a5f94bf0be210da76b9acc51 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:30.696123 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8252ce6-165d-45ff-856e-1a50a985f7e5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:30.696416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8252ce6-165d-45ff-856e-1a50a985f7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:30.696699 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8252ce6-165d-45ff-856e-1a50a985f7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:30.715936 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d8252ce6-165d-45ff-856e-1a50a985f7e5 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:30.717159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d8252ce6-165d-45ff-856e-1a50a985f7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:30.789070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 972/1950] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:30.801123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3dad1-2fc2-4532-9972-1b4af659cae8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:30.802460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fca3dad1-2fc2-4532-9972-1b4af659cae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:30.851256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:30.865265 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3dad1-2fc2-4532-9972-1b4af659cae8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:30.868887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849661a4-adff-42ab-959f-7ced4809228c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:30.869538 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849661a4-adff-42ab-959f-7ced4809228c service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:30.869826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849661a4-adff-42ab-959f-7ced4809228c service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:30.870105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849661a4-adff-42ab-959f-7ced4809228c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:30.881442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849661a4-adff-42ab-959f-7ced4809228c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:30.881442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849661a4-adff-42ab-959f-7ced4809228c service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:30.891429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 979/1951] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:31.320550 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8252ce6-165d-45ff-856e-1a50a985f7e5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:31.322175 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d8252ce6-165d-45ff-856e-1a50a985f7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:31.336381 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8252ce6-165d-45ff-856e-1a50a985f7e5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:31.341694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941ee25-6df9-4186-920e-6946cbc5ba53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:31.342591 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941ee25-6df9-4186-920e-6946cbc5ba53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:31.342890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941ee25-6df9-4186-920e-6946cbc5ba53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:31.343161 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941ee25-6df9-4186-920e-6946cbc5ba53 admin admin] PATH_INFO: `/v3/projects/55258f2339d14a68ad4b0b19e1922e7c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:31.344477 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941ee25-6df9-4186-920e-6946cbc5ba53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55258f2339d14a68ad4b0b19e1922e7c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:31.346979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941ee25-6df9-4186-920e-6946cbc5ba53 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:31.504713 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6941ee25-6df9-4186-920e-6946cbc5ba53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55258f2339d14a68ad4b0b19e1922e7c', 'request_id': 'req-6941ee25-6df9-4186-920e-6946cbc5ba53'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:31.512856 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6941ee25-6df9-4186-920e-6946cbc5ba53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55258f2339d14a68ad4b0b19e1922e7c', 'request_id': 'req-6941ee25-6df9-4186-920e-6946cbc5ba53'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:31.587415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6941ee25-6df9-4186-920e-6946cbc5ba53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55258f2339d14a68ad4b0b19e1922e7c', 'request_id': 'req-6941ee25-6df9-4186-920e-6946cbc5ba53'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:31.600654 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6941ee25-6df9-4186-920e-6946cbc5ba53 admin admin] The token cache is being invalidate because project 55258f2339d14a68ad4b0b19e1922e7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:31.601062 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6941ee25-6df9-4186-920e-6946cbc5ba53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:31.601817 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6941ee25-6df9-4186-920e-6946cbc5ba53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:31.603705 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 973/1952] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:31 2022] DELETE /identity/v3/projects/55258f2339d14a68ad4b0b19e1922e7c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:31.607754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849661a4-adff-42ab-959f-7ced4809228c service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:31.619834 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849661a4-adff-42ab-959f-7ced4809228c service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:31.661187 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941ee25-6df9-4186-920e-6946cbc5ba53 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:31.663227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6941ee25-6df9-4186-920e-6946cbc5ba53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:31.711852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef02435-f4fa-44a8-95d5-eeb8e7840a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:31.712744 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef02435-f4fa-44a8-95d5-eeb8e7840a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:31.713029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef02435-f4fa-44a8-95d5-eeb8e7840a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:31.713306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef02435-f4fa-44a8-95d5-eeb8e7840a95 admin admin] PATH_INFO: `/v3/users/9110b90eb29a4d78bad72b644b7d1a5f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:31.714506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef02435-f4fa-44a8-95d5-eeb8e7840a95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9110b90eb29a4d78bad72b644b7d1a5f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:31.716672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef02435-f4fa-44a8-95d5-eeb8e7840a95 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:31.737973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:31.753195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941ee25-6df9-4186-920e-6946cbc5ba53 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:31.758026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ef693-aed5-4034-8f48-0ab827770728 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e41222b4e8f14939a71fd9eecec64cff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57d3ab94e91a491ea40f947ffb4346cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:31.758703 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ef693-aed5-4034-8f48-0ab827770728 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:31.758990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ef693-aed5-4034-8f48-0ab827770728 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:31.759309 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ef693-aed5-4034-8f48-0ab827770728 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:31.760043 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ef693-aed5-4034-8f48-0ab827770728 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:19:31.760358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ef693-aed5-4034-8f48-0ab827770728 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:31.761979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ef693-aed5-4034-8f48-0ab827770728 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:31.777664 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 974/1953] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:19:31.792560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ef693-aed5-4034-8f48-0ab827770728 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:31.819340 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ef693-aed5-4034-8f48-0ab827770728 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:31.827938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6010b9e-d98e-4ba6-a671-4a7b5a6d83cb tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e41222b4e8f14939a71fd9eecec64cff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57d3ab94e91a491ea40f947ffb4346cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:31.828617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6010b9e-d98e-4ba6-a671-4a7b5a6d83cb tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:31.829605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6010b9e-d98e-4ba6-a671-4a7b5a6d83cb tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:31.829903 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6010b9e-d98e-4ba6-a671-4a7b5a6d83cb tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:31.831418 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6010b9e-d98e-4ba6-a671-4a7b5a6d83cb tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:31.834538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6010b9e-d98e-4ba6-a671-4a7b5a6d83cb tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:31.945075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 975/1954] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:31 2022] POST /identity/v3/projects => generated 355 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:31.950968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6010b9e-d98e-4ba6-a671-4a7b5a6d83cb tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:31.959126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bef02435-f4fa-44a8-95d5-eeb8e7840a95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9110b90eb29a4d78bad72b644b7d1a5f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:31.976224 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bef02435-f4fa-44a8-95d5-eeb8e7840a95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9110b90eb29a4d78bad72b644b7d1a5f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:32.041303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6010b9e-d98e-4ba6-a671-4a7b5a6d83cb tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:32.046591 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c663c-b99c-4af8-9670-ed91631f9300 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e41222b4e8f14939a71fd9eecec64cff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57d3ab94e91a491ea40f947ffb4346cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:32.047407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c663c-b99c-4af8-9670-ed91631f9300 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:32.047701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c663c-b99c-4af8-9670-ed91631f9300 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:32.048159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c663c-b99c-4af8-9670-ed91631f9300 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:32.050982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bef02435-f4fa-44a8-95d5-eeb8e7840a95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9110b90eb29a4d78bad72b644b7d1a5f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:32.051694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bef02435-f4fa-44a8-95d5-eeb8e7840a95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9110b90eb29a4d78bad72b644b7d1a5f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:32.054702 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c663c-b99c-4af8-9670-ed91631f9300 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:19:32.055937 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c663c-b99c-4af8-9670-ed91631f9300 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:32.061358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c663c-b99c-4af8-9670-ed91631f9300 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:32.063018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bef02435-f4fa-44a8-95d5-eeb8e7840a95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9110b90eb29a4d78bad72b644b7d1a5f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:32.072177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 980/1955] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:31 2022] DELETE /identity/v3/users/9110b90eb29a4d78bad72b644b7d1a5f => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:32.077113 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 976/1956] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:19:32.129571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef02435-f4fa-44a8-95d5-eeb8e7840a95 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:32.134029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bef02435-f4fa-44a8-95d5-eeb8e7840a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:32.213905 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef02435-f4fa-44a8-95d5-eeb8e7840a95 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:32.219270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c14003-f1a4-4fa7-b054-247d850e8c53 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e41222b4e8f14939a71fd9eecec64cff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57d3ab94e91a491ea40f947ffb4346cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:32.220268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c14003-f1a4-4fa7-b054-247d850e8c53 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:32.220629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c14003-f1a4-4fa7-b054-247d850e8c53 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:32.220974 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c14003-f1a4-4fa7-b054-247d850e8c53 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] PATH_INFO: `/v3/projects/44bf09731c9e4b10a413d49c3c0e174b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:32.223169 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c14003-f1a4-4fa7-b054-247d850e8c53 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] RBAC: Authorizing `identity:delete_project(project_id=44bf09731c9e4b10a413d49c3c0e174b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:32.225303 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c14003-f1a4-4fa7-b054-247d850e8c53 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:32.228178 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:32.395241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5c14003-f1a4-4fa7-b054-247d850e8c53 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44bf09731c9e4b10a413d49c3c0e174b', 'request_id': 'req-c5c14003-f1a4-4fa7-b054-247d850e8c53'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:32.409733 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5c14003-f1a4-4fa7-b054-247d850e8c53 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44bf09731c9e4b10a413d49c3c0e174b', 'request_id': 'req-c5c14003-f1a4-4fa7-b054-247d850e8c53'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:32.506545 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5c14003-f1a4-4fa7-b054-247d850e8c53 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44bf09731c9e4b10a413d49c3c0e174b', 'request_id': 'req-c5c14003-f1a4-4fa7-b054-247d850e8c53'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:32.518668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5c14003-f1a4-4fa7-b054-247d850e8c53 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] The token cache is being invalidate because project 44bf09731c9e4b10a413d49c3c0e174b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:32.518967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5c14003-f1a4-4fa7-b054-247d850e8c53 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:32.519645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5c14003-f1a4-4fa7-b054-247d850e8c53 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:32.524897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 981/1957] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:32 2022] DELETE /identity/v3/projects/44bf09731c9e4b10a413d49c3c0e174b => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:32.660991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c663c-b99c-4af8-9670-ed91631f9300 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:32.664977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1e2c663c-b99c-4af8-9670-ed91631f9300 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:32.799145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c663c-b99c-4af8-9670-ed91631f9300 tempest-VolumeQuotaClassesTest-1584930453 tempest-VolumeQuotaClassesTest-1584930453-project-admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:32.861172 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b3037-d06b-41cb-93fb-15e4de3d5e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:32.862811 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b3037-d06b-41cb-93fb-15e4de3d5e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:32.863096 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b3037-d06b-41cb-93fb-15e4de3d5e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:32.863421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b3037-d06b-41cb-93fb-15e4de3d5e5b admin admin] PATH_INFO: `/v3/projects/da35aa256dae43a3828f03bb1e8492bc` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:32.864692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b3037-d06b-41cb-93fb-15e4de3d5e5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=da35aa256dae43a3828f03bb1e8492bc)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:32.866935 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b3037-d06b-41cb-93fb-15e4de3d5e5b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:32.881313 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:32.988470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-210b3037-d06b-41cb-93fb-15e4de3d5e5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da35aa256dae43a3828f03bb1e8492bc', 'request_id': 'req-210b3037-d06b-41cb-93fb-15e4de3d5e5b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:32.994871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-210b3037-d06b-41cb-93fb-15e4de3d5e5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da35aa256dae43a3828f03bb1e8492bc', 'request_id': 'req-210b3037-d06b-41cb-93fb-15e4de3d5e5b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:33.072496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-210b3037-d06b-41cb-93fb-15e4de3d5e5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da35aa256dae43a3828f03bb1e8492bc', 'request_id': 'req-210b3037-d06b-41cb-93fb-15e4de3d5e5b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:33.101685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-210b3037-d06b-41cb-93fb-15e4de3d5e5b admin admin] The token cache is being invalidate because project da35aa256dae43a3828f03bb1e8492bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:33.101995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-210b3037-d06b-41cb-93fb-15e4de3d5e5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:33.102726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-210b3037-d06b-41cb-93fb-15e4de3d5e5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:33.105713 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 977/1958] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:32 2022] DELETE /identity/v3/projects/da35aa256dae43a3828f03bb1e8492bc => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:33.116981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66540a91-5317-4d9b-9ccf-2657534bf266 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:33.117488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66540a91-5317-4d9b-9ccf-2657534bf266 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:33.117997 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66540a91-5317-4d9b-9ccf-2657534bf266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:33.138152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-66540a91-5317-4d9b-9ccf-2657534bf266 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:33.139563 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-66540a91-5317-4d9b-9ccf-2657534bf266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:33.223818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 982/1959] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:33.231900 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7794af-c696-42bf-96f3-0b795c84a30a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:33.232196 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7794af-c696-42bf-96f3-0b795c84a30a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:33.232483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7794af-c696-42bf-96f3-0b795c84a30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:33.265470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5b7794af-c696-42bf-96f3-0b795c84a30a None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:33.268117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5b7794af-c696-42bf-96f3-0b795c84a30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:33.281041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 978/1960] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:33.282267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66540a91-5317-4d9b-9ccf-2657534bf266 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:33.294561 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66540a91-5317-4d9b-9ccf-2657534bf266 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:33.301644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d9cb4-db6b-41d2-9c19-9d324511642d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:33.303001 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d9cb4-db6b-41d2-9c19-9d324511642d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:33.303001 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d9cb4-db6b-41d2-9c19-9d324511642d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:33.303229 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d9cb4-db6b-41d2-9c19-9d324511642d admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:33.314671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d9cb4-db6b-41d2-9c19-9d324511642d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:19:33.314671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d9cb4-db6b-41d2-9c19-9d324511642d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:33.314671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d9cb4-db6b-41d2-9c19-9d324511642d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:33.315937 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7794af-c696-42bf-96f3-0b795c84a30a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:33.315937 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5b7794af-c696-42bf-96f3-0b795c84a30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:33.325797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7794af-c696-42bf-96f3-0b795c84a30a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:33.333041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 983/1961] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:19:33.334538 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d9cb4-db6b-41d2-9c19-9d324511642d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:33.338348 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277994b-69bf-4c37-acf1-d492aca32629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:33.340082 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277994b-69bf-4c37-acf1-d492aca32629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:33.340429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277994b-69bf-4c37-acf1-d492aca32629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:33.342946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277994b-69bf-4c37-acf1-d492aca32629 admin admin] PATH_INFO: `/v3/users/3cedcad33cbe403db994702c2da6b7f0` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:33.343247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277994b-69bf-4c37-acf1-d492aca32629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cedcad33cbe403db994702c2da6b7f0)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:33.345740 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277994b-69bf-4c37-acf1-d492aca32629 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:33.353009 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d9cb4-db6b-41d2-9c19-9d324511642d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:33.366027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71a3e6-926e-477d-869c-1b8cd149e7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:33.366716 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71a3e6-926e-477d-869c-1b8cd149e7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:33.367002 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71a3e6-926e-477d-869c-1b8cd149e7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:33.367320 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71a3e6-926e-477d-869c-1b8cd149e7d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:33.368028 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f71a3e6-926e-477d-869c-1b8cd149e7d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:33.370091 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f71a3e6-926e-477d-869c-1b8cd149e7d9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:33.456367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 984/1962] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:33 2022] POST /identity/v3/projects => generated 369 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:33.460058 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71a3e6-926e-477d-869c-1b8cd149e7d9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:33.556338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:33.560601 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71a3e6-926e-477d-869c-1b8cd149e7d9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:33.563913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e277994b-69bf-4c37-acf1-d492aca32629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cedcad33cbe403db994702c2da6b7f0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:33.569483 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2845a-6eec-4cf3-a09b-d6e1214f89c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:33.570378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2845a-6eec-4cf3-a09b-d6e1214f89c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:33.570767 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2845a-6eec-4cf3-a09b-d6e1214f89c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:33.571150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2845a-6eec-4cf3-a09b-d6e1214f89c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:33.572170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2845a-6eec-4cf3-a09b-d6e1214f89c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:33.578279 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2845a-6eec-4cf3-a09b-d6e1214f89c2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:33.597882 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e277994b-69bf-4c37-acf1-d492aca32629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cedcad33cbe403db994702c2da6b7f0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:33.598678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e277994b-69bf-4c37-acf1-d492aca32629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cedcad33cbe403db994702c2da6b7f0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:33.613694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e277994b-69bf-4c37-acf1-d492aca32629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cedcad33cbe403db994702c2da6b7f0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:33.614892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e277994b-69bf-4c37-acf1-d492aca32629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cedcad33cbe403db994702c2da6b7f0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:33.708389 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 985/1963] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:33 2022] POST /identity/v3/users => generated 285 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:33.708989 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 979/1964] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:33 2022] DELETE /identity/v3/users/3cedcad33cbe403db994702c2da6b7f0 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:33.712050 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2845a-6eec-4cf3-a09b-d6e1214f89c2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:33.713808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f48844-5f4f-491f-ba2f-a2f7d038dda9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:33.714098 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f48844-5f4f-491f-ba2f-a2f7d038dda9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:33.714378 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f48844-5f4f-491f-ba2f-a2f7d038dda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:33.716954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e5f2845a-6eec-4cf3-a09b-d6e1214f89c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:33.729941 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2845a-6eec-4cf3-a09b-d6e1214f89c2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:33.734620 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-22f48844-5f4f-491f-ba2f-a2f7d038dda9 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:33.735993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-22f48844-5f4f-491f-ba2f-a2f7d038dda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:33.821338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae6c88-9d2d-4eae-80d1-9dcbff400e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:33.822630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae6c88-9d2d-4eae-80d1-9dcbff400e75 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:33.823703 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae6c88-9d2d-4eae-80d1-9dcbff400e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:33.824013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae6c88-9d2d-4eae-80d1-9dcbff400e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:33.824853 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae6c88-9d2d-4eae-80d1-9dcbff400e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:33.827122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae6c88-9d2d-4eae-80d1-9dcbff400e75 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:33.885708 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 986/1965] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:33.895879 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae6c88-9d2d-4eae-80d1-9dcbff400e75 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:33.900552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 980/1966] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:33.918368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae6c88-9d2d-4eae-80d1-9dcbff400e75 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:33.925435 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1421d-abe5-4c80-81d8-423a0119e263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:33.926520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1421d-abe5-4c80-81d8-423a0119e263 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:33.927030 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1421d-abe5-4c80-81d8-423a0119e263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:33.928869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1421d-abe5-4c80-81d8-423a0119e263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:33.929899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1421d-abe5-4c80-81d8-423a0119e263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:33.932993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1421d-abe5-4c80-81d8-423a0119e263 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:33.952878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f48844-5f4f-491f-ba2f-a2f7d038dda9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:33.959295 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 987/1967] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:19:33.959680 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:33.966981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f48844-5f4f-491f-ba2f-a2f7d038dda9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:33.974638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b7d39-69a4-427a-be83-526aa0b8d4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:33.975844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b7d39-69a4-427a-be83-526aa0b8d4ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:33.976198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b7d39-69a4-427a-be83-526aa0b8d4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:33.976657 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b7d39-69a4-427a-be83-526aa0b8d4ee admin admin] PATH_INFO: `/v3/projects/77496a24e01e461986a18995f456c10c/users/46aa9e9140014bc3b555de7f1e2b1df4/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:33.986596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1421d-abe5-4c80-81d8-423a0119e263 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:33.988947 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f5f1421d-abe5-4c80-81d8-423a0119e263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:34.020993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b7d39-69a4-427a-be83-526aa0b8d4ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=77496a24e01e461986a18995f456c10c, user_id=46aa9e9140014bc3b555de7f1e2b1df4, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:34.024299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b7d39-69a4-427a-be83-526aa0b8d4ee admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:34.068620 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1421d-abe5-4c80-81d8-423a0119e263 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:34.073206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4361da-ae49-473a-a1be-22e9b37ae47e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:34.073989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4361da-ae49-473a-a1be-22e9b37ae47e service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:34.074380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4361da-ae49-473a-a1be-22e9b37ae47e service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:34.074724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4361da-ae49-473a-a1be-22e9b37ae47e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:34.090195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4361da-ae49-473a-a1be-22e9b37ae47e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:34.091327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4361da-ae49-473a-a1be-22e9b37ae47e service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:34.099499 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 981/1968] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:33 2022] PUT /identity/v3/projects/77496a24e01e461986a18995f456c10c/users/46aa9e9140014bc3b555de7f1e2b1df4/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:34.110276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13adc3-dea8-46bf-b7e1-1bdad258abde None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:34.111155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13adc3-dea8-46bf-b7e1-1bdad258abde None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:34.111717 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13adc3-dea8-46bf-b7e1-1bdad258abde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:34.115785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 988/1969] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:34.142422 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-fe13adc3-dea8-46bf-b7e1-1bdad258abde None None] MFA Rules not processed for user `46aa9e9140014bc3b555de7f1e2b1df4`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:34.144619 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fe13adc3-dea8-46bf-b7e1-1bdad258abde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:34.247015 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 982/1970] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:34.259936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4361da-ae49-473a-a1be-22e9b37ae47e service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:34.261401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5a4361da-ae49-473a-a1be-22e9b37ae47e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:34.339842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4361da-ae49-473a-a1be-22e9b37ae47e service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:34.378372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:34.398182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60c7bc-05c6-463b-b7ed-66450d2dd0aa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1971c464293f426f97d6cb199e48193b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:34.398906 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60c7bc-05c6-463b-b7ed-66450d2dd0aa service glance] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:34.399249 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60c7bc-05c6-463b-b7ed-66450d2dd0aa service glance] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:34.399536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60c7bc-05c6-463b-b7ed-66450d2dd0aa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:34.408774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60c7bc-05c6-463b-b7ed-66450d2dd0aa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:34.410393 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60c7bc-05c6-463b-b7ed-66450d2dd0aa service glance] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:34.419926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 989/1971] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:34 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:34.720442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13adc3-dea8-46bf-b7e1-1bdad258abde None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:34.723037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fe13adc3-dea8-46bf-b7e1-1bdad258abde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:34.804648 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13adc3-dea8-46bf-b7e1-1bdad258abde None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:34.809448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647f8fd-1c98-4d6d-bb31-6a003455be45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:34.810346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f8fd-1c98-4d6d-bb31-6a003455be45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:34.810764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f8fd-1c98-4d6d-bb31-6a003455be45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:34.811044 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f8fd-1c98-4d6d-bb31-6a003455be45 admin admin] PATH_INFO: `/v3/users/47a883838ce74582be4272143fdee208` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:34.812367 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647f8fd-1c98-4d6d-bb31-6a003455be45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47a883838ce74582be4272143fdee208)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:34.814420 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647f8fd-1c98-4d6d-bb31-6a003455be45 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:34.938375 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60c7bc-05c6-463b-b7ed-66450d2dd0aa service glance] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:34.941456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6f60c7bc-05c6-463b-b7ed-66450d2dd0aa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:34.962607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60c7bc-05c6-463b-b7ed-66450d2dd0aa service glance] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:34.969031 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486f3d5-c35b-4686-ba81-4d5808c17b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:34.970184 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486f3d5-c35b-4686-ba81-4d5808c17b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:34.970184 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486f3d5-c35b-4686-ba81-4d5808c17b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:34.970435 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486f3d5-c35b-4686-ba81-4d5808c17b76 admin admin] PATH_INFO: `/v3/projects/86b3ba387f7d4daca0f6ffbbeff58b29` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:34.972830 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486f3d5-c35b-4686-ba81-4d5808c17b76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86b3ba387f7d4daca0f6ffbbeff58b29)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:34.976797 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486f3d5-c35b-4686-ba81-4d5808c17b76 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:35.106430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c647f8fd-1c98-4d6d-bb31-6a003455be45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47a883838ce74582be4272143fdee208'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:35.121224 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d486f3d5-c35b-4686-ba81-4d5808c17b76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86b3ba387f7d4daca0f6ffbbeff58b29', 'request_id': 'req-d486f3d5-c35b-4686-ba81-4d5808c17b76'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:35.122483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c647f8fd-1c98-4d6d-bb31-6a003455be45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47a883838ce74582be4272143fdee208'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:35.123596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c647f8fd-1c98-4d6d-bb31-6a003455be45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47a883838ce74582be4272143fdee208'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:35.129727 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:35.132610 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d486f3d5-c35b-4686-ba81-4d5808c17b76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86b3ba387f7d4daca0f6ffbbeff58b29', 'request_id': 'req-d486f3d5-c35b-4686-ba81-4d5808c17b76'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:35.133949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c647f8fd-1c98-4d6d-bb31-6a003455be45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47a883838ce74582be4272143fdee208'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:35.135026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c647f8fd-1c98-4d6d-bb31-6a003455be45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47a883838ce74582be4272143fdee208'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:35.222458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d486f3d5-c35b-4686-ba81-4d5808c17b76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86b3ba387f7d4daca0f6ffbbeff58b29', 'request_id': 'req-d486f3d5-c35b-4686-ba81-4d5808c17b76'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:35.237415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d486f3d5-c35b-4686-ba81-4d5808c17b76 admin admin] The token cache is being invalidate because project 86b3ba387f7d4daca0f6ffbbeff58b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:35.237934 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d486f3d5-c35b-4686-ba81-4d5808c17b76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:35.238770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d486f3d5-c35b-4686-ba81-4d5808c17b76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:35.240888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 990/1972] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:34 2022] DELETE /identity/v3/projects/86b3ba387f7d4daca0f6ffbbeff58b29 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:35.244594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486f3d5-c35b-4686-ba81-4d5808c17b76 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:35.246075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d486f3d5-c35b-4686-ba81-4d5808c17b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:35.302115 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 983/1973] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:34 2022] DELETE /identity/v3/users/47a883838ce74582be4272143fdee208 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:35.308092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d06c4b-5d73-4c58-8728-bbd3e164dc86 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:35.308393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d06c4b-5d73-4c58-8728-bbd3e164dc86 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:35.308756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d06c4b-5d73-4c58-8728-bbd3e164dc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:35.328700 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-89d06c4b-5d73-4c58-8728-bbd3e164dc86 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:35.329991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-89d06c4b-5d73-4c58-8728-bbd3e164dc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:35.334762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486f3d5-c35b-4686-ba81-4d5808c17b76 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:35.479061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 984/1974] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:35.484849 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bddb7-18f8-4897-90c7-ce1fe1d4c671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:35.487725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bddb7-18f8-4897-90c7-ce1fe1d4c671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:35.488044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bddb7-18f8-4897-90c7-ce1fe1d4c671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:35.488418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bddb7-18f8-4897-90c7-ce1fe1d4c671 admin admin] PATH_INFO: `/v3/users/e41222b4e8f14939a71fd9eecec64cff` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:35.492044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82bddb7-18f8-4897-90c7-ce1fe1d4c671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e41222b4e8f14939a71fd9eecec64cff)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:35.495804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d06c4b-5d73-4c58-8728-bbd3e164dc86 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:35.498137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-89d06c4b-5d73-4c58-8728-bbd3e164dc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:35.505791 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82bddb7-18f8-4897-90c7-ce1fe1d4c671 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:35.520743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:35.608984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d06c4b-5d73-4c58-8728-bbd3e164dc86 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:35.614240 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33209726-e7ec-4c73-96e8-b9733afb2480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:35.615208 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33209726-e7ec-4c73-96e8-b9733afb2480 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:35.615519 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33209726-e7ec-4c73-96e8-b9733afb2480 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:35.615799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33209726-e7ec-4c73-96e8-b9733afb2480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:35.626723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33209726-e7ec-4c73-96e8-b9733afb2480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:35.628790 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33209726-e7ec-4c73-96e8-b9733afb2480 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:35.639693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 985/1975] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:35.763020 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e82bddb7-18f8-4897-90c7-ce1fe1d4c671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e41222b4e8f14939a71fd9eecec64cff'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:35.804925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e82bddb7-18f8-4897-90c7-ce1fe1d4c671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e41222b4e8f14939a71fd9eecec64cff'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:35.889897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e82bddb7-18f8-4897-90c7-ce1fe1d4c671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e41222b4e8f14939a71fd9eecec64cff'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:35.890547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e82bddb7-18f8-4897-90c7-ce1fe1d4c671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e41222b4e8f14939a71fd9eecec64cff'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:35.902822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e82bddb7-18f8-4897-90c7-ce1fe1d4c671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e41222b4e8f14939a71fd9eecec64cff'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:35.922816 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 991/1976] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:35 2022] DELETE /identity/v3/users/e41222b4e8f14939a71fd9eecec64cff => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:36.252773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33209726-e7ec-4c73-96e8-b9733afb2480 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:36.254360 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-33209726-e7ec-4c73-96e8-b9733afb2480 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:36.270929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33209726-e7ec-4c73-96e8-b9733afb2480 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:36.359003 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186d2bf-42f1-4e68-85be-800454dfd7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:36.359905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186d2bf-42f1-4e68-85be-800454dfd7bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:36.360191 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186d2bf-42f1-4e68-85be-800454dfd7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:36.360472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186d2bf-42f1-4e68-85be-800454dfd7bb admin admin] PATH_INFO: `/v3/projects/db09a8c1aa9c45848310796420c9233c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:36.362053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186d2bf-42f1-4e68-85be-800454dfd7bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=db09a8c1aa9c45848310796420c9233c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:36.364502 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186d2bf-42f1-4e68-85be-800454dfd7bb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:36.473012 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:36.485907 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f186d2bf-42f1-4e68-85be-800454dfd7bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db09a8c1aa9c45848310796420c9233c', 'request_id': 'req-f186d2bf-42f1-4e68-85be-800454dfd7bb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:36.493529 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f186d2bf-42f1-4e68-85be-800454dfd7bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db09a8c1aa9c45848310796420c9233c', 'request_id': 'req-f186d2bf-42f1-4e68-85be-800454dfd7bb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:36.520986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bddb7-18f8-4897-90c7-ce1fe1d4c671 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:36.522512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e82bddb7-18f8-4897-90c7-ce1fe1d4c671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:36.536375 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bddb7-18f8-4897-90c7-ce1fe1d4c671 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:36.578405 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f186d2bf-42f1-4e68-85be-800454dfd7bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db09a8c1aa9c45848310796420c9233c', 'request_id': 'req-f186d2bf-42f1-4e68-85be-800454dfd7bb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:36.603717 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f186d2bf-42f1-4e68-85be-800454dfd7bb admin admin] The token cache is being invalidate because project db09a8c1aa9c45848310796420c9233c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:36.603949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f186d2bf-42f1-4e68-85be-800454dfd7bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:36.604696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f186d2bf-42f1-4e68-85be-800454dfd7bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:36.606520 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 986/1977] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:36 2022] DELETE /identity/v3/projects/db09a8c1aa9c45848310796420c9233c => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:36.608096 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b616b8b-4525-4882-a0d4-ef0b85ea3f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:36.612246 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b616b8b-4525-4882-a0d4-ef0b85ea3f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:36.612246 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b616b8b-4525-4882-a0d4-ef0b85ea3f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:36.612246 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b616b8b-4525-4882-a0d4-ef0b85ea3f28 admin admin] PATH_INFO: `/v3/projects/57d3ab94e91a491ea40f947ffb4346cb` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:36.612246 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b616b8b-4525-4882-a0d4-ef0b85ea3f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57d3ab94e91a491ea40f947ffb4346cb)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:36.614269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b616b8b-4525-4882-a0d4-ef0b85ea3f28 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:36.615262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186d2bf-42f1-4e68-85be-800454dfd7bb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:36.616863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f186d2bf-42f1-4e68-85be-800454dfd7bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:36.631997 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186d2bf-42f1-4e68-85be-800454dfd7bb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:36.638025 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb01876-2f25-4562-beac-e6b4264da518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:36.638953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb01876-2f25-4562-beac-e6b4264da518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:36.639277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb01876-2f25-4562-beac-e6b4264da518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:36.639600 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb01876-2f25-4562-beac-e6b4264da518 admin admin] PATH_INFO: `/v3/users/43581f76c4d540108494a84337dcb152` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:36.640968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb01876-2f25-4562-beac-e6b4264da518 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43581f76c4d540108494a84337dcb152)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:36.643554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb01876-2f25-4562-beac-e6b4264da518 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:36.713090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0b616b8b-4525-4882-a0d4-ef0b85ea3f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57d3ab94e91a491ea40f947ffb4346cb', 'request_id': 'req-0b616b8b-4525-4882-a0d4-ef0b85ea3f28'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:36.719268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0b616b8b-4525-4882-a0d4-ef0b85ea3f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57d3ab94e91a491ea40f947ffb4346cb', 'request_id': 'req-0b616b8b-4525-4882-a0d4-ef0b85ea3f28'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:36.810911 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0b616b8b-4525-4882-a0d4-ef0b85ea3f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57d3ab94e91a491ea40f947ffb4346cb', 'request_id': 'req-0b616b8b-4525-4882-a0d4-ef0b85ea3f28'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:36.823810 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7eb01876-2f25-4562-beac-e6b4264da518 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43581f76c4d540108494a84337dcb152'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:36.834196 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0b616b8b-4525-4882-a0d4-ef0b85ea3f28 admin admin] The token cache is being invalidate because project 57d3ab94e91a491ea40f947ffb4346cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:36.839834 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0b616b8b-4525-4882-a0d4-ef0b85ea3f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:36.839834 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0b616b8b-4525-4882-a0d4-ef0b85ea3f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:36.840281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 992/1978] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:36 2022] DELETE /identity/v3/projects/57d3ab94e91a491ea40f947ffb4346cb => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:36.844918 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89568d39-c7b8-4ee9-b740-7b0a6b7aae8e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:36.845290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89568d39-c7b8-4ee9-b740-7b0a6b7aae8e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:36.845648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89568d39-c7b8-4ee9-b740-7b0a6b7aae8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:36.849548 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7eb01876-2f25-4562-beac-e6b4264da518 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43581f76c4d540108494a84337dcb152'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:36.850271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7eb01876-2f25-4562-beac-e6b4264da518 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43581f76c4d540108494a84337dcb152'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:36.868701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7eb01876-2f25-4562-beac-e6b4264da518 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43581f76c4d540108494a84337dcb152'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:36.868996 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-89568d39-c7b8-4ee9-b740-7b0a6b7aae8e None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:36.869079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7eb01876-2f25-4562-beac-e6b4264da518 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43581f76c4d540108494a84337dcb152'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:36.869153 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-89568d39-c7b8-4ee9-b740-7b0a6b7aae8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:36.877578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:36.925400 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 987/1979] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:36 2022] DELETE /identity/v3/users/43581f76c4d540108494a84337dcb152 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:36.945398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 993/1980] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:36.953766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ea88f-11e9-44c1-ac91-58b4ae237d18 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:36.955113 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ea88f-11e9-44c1-ac91-58b4ae237d18 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:36.955987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ea88f-11e9-44c1-ac91-58b4ae237d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:36.976536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e78ea88f-11e9-44c1-ac91-58b4ae237d18 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:36.977797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e78ea88f-11e9-44c1-ac91-58b4ae237d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:37.043333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 988/1981] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:37.047288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89568d39-c7b8-4ee9-b740-7b0a6b7aae8e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:37.068354 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89568d39-c7b8-4ee9-b740-7b0a6b7aae8e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:37.074320 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd0f11-88fe-42ea-a309-d41d3b5f3947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:37.075109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd0f11-88fe-42ea-a309-d41d3b5f3947 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:37.075486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd0f11-88fe-42ea-a309-d41d3b5f3947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:37.076058 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd0f11-88fe-42ea-a309-d41d3b5f3947 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:37.076877 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd0f11-88fe-42ea-a309-d41d3b5f3947 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:19:37.077263 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd0f11-88fe-42ea-a309-d41d3b5f3947 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:37.078987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd0f11-88fe-42ea-a309-d41d3b5f3947 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:37.105503 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 994/1982] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:19:37.165310 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ea88f-11e9-44c1-ac91-58b4ae237d18 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:37.175368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ea88f-11e9-44c1-ac91-58b4ae237d18 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:37.180028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d95bbb-982e-4d98-b3a9-e9c2bf96476d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:37.180684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d95bbb-982e-4d98-b3a9-e9c2bf96476d admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:37.180966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d95bbb-982e-4d98-b3a9-e9c2bf96476d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:37.181239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d95bbb-982e-4d98-b3a9-e9c2bf96476d admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:37.181935 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d95bbb-982e-4d98-b3a9-e9c2bf96476d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:37.183908 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d95bbb-982e-4d98-b3a9-e9c2bf96476d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:37.317083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 989/1983] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:37 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:37.321116 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd0f11-88fe-42ea-a309-d41d3b5f3947 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:37.331881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd0f11-88fe-42ea-a309-d41d3b5f3947 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:37.391290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbc61d-694b-48f7-8d72-a3183ae51ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:37.400703 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbc61d-694b-48f7-8d72-a3183ae51ccd admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:37.400703 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbc61d-694b-48f7-8d72-a3183ae51ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:37.400703 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbc61d-694b-48f7-8d72-a3183ae51ccd admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:37.400703 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbc61d-694b-48f7-8d72-a3183ae51ccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:37.400703 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbc61d-694b-48f7-8d72-a3183ae51ccd admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:37.690594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 995/1984] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:37 2022] POST /identity/v3/users => generated 294 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:37.695157 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d95bbb-982e-4d98-b3a9-e9c2bf96476d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:37.706385 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d95bbb-982e-4d98-b3a9-e9c2bf96476d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:37.711133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1cb9be-8221-4844-ab25-513811728640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:37.711913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cb9be-8221-4844-ab25-513811728640 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:37.712292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cb9be-8221-4844-ab25-513811728640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:37.712648 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cb9be-8221-4844-ab25-513811728640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:37.713531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1cb9be-8221-4844-ab25-513811728640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:37.715215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1cb9be-8221-4844-ab25-513811728640 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:37.722358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbc61d-694b-48f7-8d72-a3183ae51ccd admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:37.724685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d1cbc61d-694b-48f7-8d72-a3183ae51ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:37.726615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:37.739083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbc61d-694b-48f7-8d72-a3183ae51ccd admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:37.742866 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 990/1985] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:37.743681 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1cb9be-8221-4844-ab25-513811728640 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:37.746292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e7d19-8619-449e-873d-d595a0c9a931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:37.747343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e7d19-8619-449e-873d-d595a0c9a931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:37.748190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e7d19-8619-449e-873d-d595a0c9a931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:37.748190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e7d19-8619-449e-873d-d595a0c9a931 admin admin] PATH_INFO: `/v3/projects/d4750ba043044c8280e8e0239df72a4e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:37.749573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e7d19-8619-449e-873d-d595a0c9a931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4750ba043044c8280e8e0239df72a4e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:37.757502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e7d19-8619-449e-873d-d595a0c9a931 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:37.762927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1cb9be-8221-4844-ab25-513811728640 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:37.770237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59157795-f553-4638-bee1-791efc444c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:37.771028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59157795-f553-4638-bee1-791efc444c4b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:37.771459 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59157795-f553-4638-bee1-791efc444c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:37.771838 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59157795-f553-4638-bee1-791efc444c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:37.772704 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59157795-f553-4638-bee1-791efc444c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:37.774402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59157795-f553-4638-bee1-791efc444c4b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:37.786911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 991/1986] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:37.792167 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59157795-f553-4638-bee1-791efc444c4b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:37.803366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59157795-f553-4638-bee1-791efc444c4b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:37.808514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab39d3-3b1d-422d-aa8b-51aa75bcfc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:37.809400 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab39d3-3b1d-422d-aa8b-51aa75bcfc23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:37.809683 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab39d3-3b1d-422d-aa8b-51aa75bcfc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:37.809984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab39d3-3b1d-422d-aa8b-51aa75bcfc23 admin admin] PATH_INFO: `/v3/projects/d02aeaea285746ea959615c310785575/users/bf91706eba964fbe94c9817c43ef8bdd/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:37.827547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab39d3-3b1d-422d-aa8b-51aa75bcfc23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d02aeaea285746ea959615c310785575, user_id=bf91706eba964fbe94c9817c43ef8bdd, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:37.830927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab39d3-3b1d-422d-aa8b-51aa75bcfc23 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:37.903089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 992/1987] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:37 2022] PUT /identity/v3/projects/d02aeaea285746ea959615c310785575/users/bf91706eba964fbe94c9817c43ef8bdd/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:37.914134 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa1321-2bc3-4c5e-9a06-295bc550292f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:37.914444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa1321-2bc3-4c5e-9a06-295bc550292f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:37.914800 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa1321-2bc3-4c5e-9a06-295bc550292f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:37.918441 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3f1e7d19-8619-449e-873d-d595a0c9a931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4750ba043044c8280e8e0239df72a4e', 'request_id': 'req-3f1e7d19-8619-449e-873d-d595a0c9a931'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:37.926424 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3f1e7d19-8619-449e-873d-d595a0c9a931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4750ba043044c8280e8e0239df72a4e', 'request_id': 'req-3f1e7d19-8619-449e-873d-d595a0c9a931'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:37.951351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-12aa1321-2bc3-4c5e-9a06-295bc550292f None None] MFA Rules not processed for user `bf91706eba964fbe94c9817c43ef8bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:37.952587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-12aa1321-2bc3-4c5e-9a06-295bc550292f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:38.022790 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 993/1988] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:38.027459 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa1321-2bc3-4c5e-9a06-295bc550292f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:38.049521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3f1e7d19-8619-449e-873d-d595a0c9a931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4750ba043044c8280e8e0239df72a4e', 'request_id': 'req-3f1e7d19-8619-449e-873d-d595a0c9a931'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:38.064506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3f1e7d19-8619-449e-873d-d595a0c9a931 admin admin] The token cache is being invalidate because project d4750ba043044c8280e8e0239df72a4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:38.065537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3f1e7d19-8619-449e-873d-d595a0c9a931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:38.066484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3f1e7d19-8619-449e-873d-d595a0c9a931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:38.069781 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 996/1989] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:37 2022] DELETE /identity/v3/projects/d4750ba043044c8280e8e0239df72a4e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:38.083675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6364040-2f6e-42c5-b8ae-d37fc32b0d73 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.083963 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6364040-2f6e-42c5-b8ae-d37fc32b0d73 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.084239 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6364040-2f6e-42c5-b8ae-d37fc32b0d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.098659 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa1321-2bc3-4c5e-9a06-295bc550292f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:38.104200 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ddb6b1-309d-4b81-b58e-87ef60f21d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:38.105000 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddb6b1-309d-4b81-b58e-87ef60f21d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.105847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddb6b1-309d-4b81-b58e-87ef60f21d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.107213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddb6b1-309d-4b81-b58e-87ef60f21d2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.107929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ddb6b1-309d-4b81-b58e-87ef60f21d2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:38.109870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ddb6b1-309d-4b81-b58e-87ef60f21d2f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:38.112486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a6364040-2f6e-42c5-b8ae-d37fc32b0d73 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:38.114647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a6364040-2f6e-42c5-b8ae-d37fc32b0d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:38.127205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 997/1990] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:38.131665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91defd01-a4b6-4768-b05b-38b04d06c7be None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.132081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91defd01-a4b6-4768-b05b-38b04d06c7be None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.132463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91defd01-a4b6-4768-b05b-38b04d06c7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.150639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:38.157007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-91defd01-a4b6-4768-b05b-38b04d06c7be None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:38.158378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-91defd01-a4b6-4768-b05b-38b04d06c7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:38.177462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 998/1991] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:38.178893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91defd01-a4b6-4768-b05b-38b04d06c7be None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:38.189200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91defd01-a4b6-4768-b05b-38b04d06c7be None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:38.200411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3b1fa-87f1-40d6-bbbb-5f144c130bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:38.201801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3b1fa-87f1-40d6-bbbb-5f144c130bfc admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.203524 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3b1fa-87f1-40d6-bbbb-5f144c130bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.204065 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3b1fa-87f1-40d6-bbbb-5f144c130bfc admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.205720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3b1fa-87f1-40d6-bbbb-5f144c130bfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:19:38.206218 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3b1fa-87f1-40d6-bbbb-5f144c130bfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:38.208049 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3b1fa-87f1-40d6-bbbb-5f144c130bfc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:38.212398 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 994/1992] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:38 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:38.213702 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ddb6b1-309d-4b81-b58e-87ef60f21d2f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:38.215615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c3ddb6b1-309d-4b81-b58e-87ef60f21d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:38.228219 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 999/1993] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:19:38.232686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3b1fa-87f1-40d6-bbbb-5f144c130bfc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:38.244330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3b1fa-87f1-40d6-bbbb-5f144c130bfc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:38.311273 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ddb6b1-309d-4b81-b58e-87ef60f21d2f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:38.317473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd090cb-ac51-418d-8a56-b85261493c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:38.318274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd090cb-ac51-418d-8a56-b85261493c76 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.318563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd090cb-ac51-418d-8a56-b85261493c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.318843 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd090cb-ac51-418d-8a56-b85261493c76 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.319572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd090cb-ac51-418d-8a56-b85261493c76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:38.321668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd090cb-ac51-418d-8a56-b85261493c76 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:38.322315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f134a-1e1c-4f06-ba81-4a125a96630b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:38.322908 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f134a-1e1c-4f06-ba81-4a125a96630b admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.323243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f134a-1e1c-4f06-ba81-4a125a96630b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.323527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f134a-1e1c-4f06-ba81-4a125a96630b admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.324223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f134a-1e1c-4f06-ba81-4a125a96630b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:38.326144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f134a-1e1c-4f06-ba81-4a125a96630b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:38.421602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 995/1994] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:38 2022] POST /identity/v3/users => generated 299 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:38.425968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd090cb-ac51-418d-8a56-b85261493c76 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:38.435855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd090cb-ac51-418d-8a56-b85261493c76 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:38.443471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356f720-58b5-40c6-8e7b-889a7d35e86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:38.444149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356f720-58b5-40c6-8e7b-889a7d35e86d admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.445087 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356f720-58b5-40c6-8e7b-889a7d35e86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.445087 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356f720-58b5-40c6-8e7b-889a7d35e86d admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.445698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356f720-58b5-40c6-8e7b-889a7d35e86d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:38.450734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1000/1995] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:38 2022] POST /identity/v3/projects => generated 387 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:38.451007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356f720-58b5-40c6-8e7b-889a7d35e86d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:38.451548 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f134a-1e1c-4f06-ba81-4a125a96630b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:38.460209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 996/1996] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:19:38.464232 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356f720-58b5-40c6-8e7b-889a7d35e86d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:38.474453 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356f720-58b5-40c6-8e7b-889a7d35e86d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:38.520381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f134a-1e1c-4f06-ba81-4a125a96630b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:38.525194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c08d2c-2f29-4e3a-9c9a-b8f5e381a3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:38.529175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c08d2c-2f29-4e3a-9c9a-b8f5e381a3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.529175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c08d2c-2f29-4e3a-9c9a-b8f5e381a3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.529175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c08d2c-2f29-4e3a-9c9a-b8f5e381a3e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.529175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c08d2c-2f29-4e3a-9c9a-b8f5e381a3e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:38.529175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c08d2c-2f29-4e3a-9c9a-b8f5e381a3e6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:38.533061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832369bb-4bc0-4cef-a9e0-f75d1d836acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:38.535281 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832369bb-4bc0-4cef-a9e0-f75d1d836acf admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.535744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832369bb-4bc0-4cef-a9e0-f75d1d836acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.536127 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832369bb-4bc0-4cef-a9e0-f75d1d836acf admin admin] PATH_INFO: `/v3/projects/b286849fbbff4d71b238ebad21903abc/users/3fa540ae0ebf4290a334fed806ff7d88/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.554230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832369bb-4bc0-4cef-a9e0-f75d1d836acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b286849fbbff4d71b238ebad21903abc, user_id=3fa540ae0ebf4290a334fed806ff7d88, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:38.556956 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832369bb-4bc0-4cef-a9e0-f75d1d836acf admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:38.616830 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1001/1997] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:38 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:38.621013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c08d2c-2f29-4e3a-9c9a-b8f5e381a3e6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:38.630677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c08d2c-2f29-4e3a-9c9a-b8f5e381a3e6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:38.635423 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b232a-9e7c-4a73-8007-402302dfaaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:38.636068 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b232a-9e7c-4a73-8007-402302dfaaff admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.636351 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b232a-9e7c-4a73-8007-402302dfaaff admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.636630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b232a-9e7c-4a73-8007-402302dfaaff admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.637422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b232a-9e7c-4a73-8007-402302dfaaff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:38.639015 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b232a-9e7c-4a73-8007-402302dfaaff admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:38.639706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:38.646631 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 997/1998] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:38 2022] PUT /identity/v3/projects/b286849fbbff4d71b238ebad21903abc/users/3fa540ae0ebf4290a334fed806ff7d88/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:38.650367 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832369bb-4bc0-4cef-a9e0-f75d1d836acf admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:38.653046 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1002/1999] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:19:38.663208 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b232a-9e7c-4a73-8007-402302dfaaff admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:38.673099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b232a-9e7c-4a73-8007-402302dfaaff admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:38.728578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832369bb-4bc0-4cef-a9e0-f75d1d836acf admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:38.733509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a925c5-1d49-4981-90e6-ee01a1ad3a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:38.735296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a03e0a-d6a4-44ed-a3b5-bf8742851b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:38.735759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a925c5-1d49-4981-90e6-ee01a1ad3a24 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.735861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a03e0a-d6a4-44ed-a3b5-bf8742851b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.735955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a925c5-1d49-4981-90e6-ee01a1ad3a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.736059 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a03e0a-d6a4-44ed-a3b5-bf8742851b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.736153 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a925c5-1d49-4981-90e6-ee01a1ad3a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.736251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a03e0a-d6a4-44ed-a3b5-bf8742851b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.736342 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a925c5-1d49-4981-90e6-ee01a1ad3a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:38.736453 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a03e0a-d6a4-44ed-a3b5-bf8742851b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:38.738019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a925c5-1d49-4981-90e6-ee01a1ad3a24 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:38.738152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a03e0a-d6a4-44ed-a3b5-bf8742851b42 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:38.748082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1003/2001] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:19:38.748478 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 998/2001] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:19:38.752137 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a03e0a-d6a4-44ed-a3b5-bf8742851b42 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:38.752649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a925c5-1d49-4981-90e6-ee01a1ad3a24 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:38.762744 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a03e0a-d6a4-44ed-a3b5-bf8742851b42 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:38.763714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a925c5-1d49-4981-90e6-ee01a1ad3a24 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:38.765546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:38.768701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98f0b5-a8a1-4fbe-ae6f-22de4608c3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:38.769982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98f0b5-a8a1-4fbe-ae6f-22de4608c3d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.770155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69528462-14a0-4d79-8d46-4da409c551ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:38.770274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98f0b5-a8a1-4fbe-ae6f-22de4608c3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.770274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98f0b5-a8a1-4fbe-ae6f-22de4608c3d2 admin admin] PATH_INFO: `/v3/projects/b286849fbbff4d71b238ebad21903abc/users/3fa540ae0ebf4290a334fed806ff7d88/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.770800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69528462-14a0-4d79-8d46-4da409c551ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.771086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69528462-14a0-4d79-8d46-4da409c551ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.771467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69528462-14a0-4d79-8d46-4da409c551ab admin admin] PATH_INFO: `/v3/projects/7b09ad2ebed44a51b1613c02d8f5be03/users/6577082bc2e6446591191a439b27de38/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.772260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98f0b5-a8a1-4fbe-ae6f-22de4608c3d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b286849fbbff4d71b238ebad21903abc, user_id=3fa540ae0ebf4290a334fed806ff7d88, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:38.774916 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98f0b5-a8a1-4fbe-ae6f-22de4608c3d2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:38.787965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69528462-14a0-4d79-8d46-4da409c551ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b09ad2ebed44a51b1613c02d8f5be03, user_id=6577082bc2e6446591191a439b27de38, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:38.792412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 999/2002] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:38 2022] PUT /identity/v3/projects/b286849fbbff4d71b238ebad21903abc/users/3fa540ae0ebf4290a334fed806ff7d88/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:38.792573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69528462-14a0-4d79-8d46-4da409c551ab admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:38.802911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1394133-81b4-4d4e-9b7a-675f86622320 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.803237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1394133-81b4-4d4e-9b7a-675f86622320 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.803534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1394133-81b4-4d4e-9b7a-675f86622320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.833073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b1394133-81b4-4d4e-9b7a-675f86622320 None None] MFA Rules not processed for user `3fa540ae0ebf4290a334fed806ff7d88`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:38.834311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b1394133-81b4-4d4e-9b7a-675f86622320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:38.845983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:38.881342 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1004/2003] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:38 2022] PUT /identity/v3/projects/7b09ad2ebed44a51b1613c02d8f5be03/users/6577082bc2e6446591191a439b27de38/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:38.892240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14b894-63c5-4e8f-ae18-57b340a52c36 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:38.892526 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14b894-63c5-4e8f-ae18-57b340a52c36 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:38.893512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14b894-63c5-4e8f-ae18-57b340a52c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:38.906732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1000/2004] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:38.918526 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1394133-81b4-4d4e-9b7a-675f86622320 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:38.920042 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b1394133-81b4-4d4e-9b7a-675f86622320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:38.932899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-de14b894-63c5-4e8f-ae18-57b340a52c36 None None] MFA Rules not processed for user `6577082bc2e6446591191a439b27de38`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:38.934319 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-de14b894-63c5-4e8f-ae18-57b340a52c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:38.979622 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1394133-81b4-4d4e-9b7a-675f86622320 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:39.007099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1005/2005] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:39.011853 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14b894-63c5-4e8f-ae18-57b340a52c36 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:39.032427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8059c-bc9d-431b-a61f-8879ff01d37e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:39.032852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8059c-bc9d-431b-a61f-8879ff01d37e service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:39.033206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8059c-bc9d-431b-a61f-8879ff01d37e service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:39.034255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8059c-bc9d-431b-a61f-8879ff01d37e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:39.075881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14b894-63c5-4e8f-ae18-57b340a52c36 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:39.081297 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760d7ca-63ef-4c41-99c0-fc99798cec33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:39.082142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760d7ca-63ef-4c41-99c0-fc99798cec33 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:39.082434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760d7ca-63ef-4c41-99c0-fc99798cec33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:39.082733 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760d7ca-63ef-4c41-99c0-fc99798cec33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:39.083470 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760d7ca-63ef-4c41-99c0-fc99798cec33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:39.085395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760d7ca-63ef-4c41-99c0-fc99798cec33 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:39.098381 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8059c-bc9d-431b-a61f-8879ff01d37e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:39.099920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8059c-bc9d-431b-a61f-8879ff01d37e service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:39.110007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1001/2006] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:19:39.178784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1006/2007] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:39 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:39.180920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8059c-bc9d-431b-a61f-8879ff01d37e service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:39.191482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8059c-bc9d-431b-a61f-8879ff01d37e service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:39.202792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:39.249996 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489af8a-137a-47da-91e8-49c5287deb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:39.250674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489af8a-137a-47da-91e8-49c5287deb10 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:39.250961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489af8a-137a-47da-91e8-49c5287deb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:39.251278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489af8a-137a-47da-91e8-49c5287deb10 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:39.251981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489af8a-137a-47da-91e8-49c5287deb10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:39.254838 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489af8a-137a-47da-91e8-49c5287deb10 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:39.359622 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1002/2008] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:39 2022] POST /identity/v3/users => generated 294 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:39.364506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760d7ca-63ef-4c41-99c0-fc99798cec33 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:39.376757 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760d7ca-63ef-4c41-99c0-fc99798cec33 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:39.384332 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8662a94-1b16-4d6c-8b34-b747aaf9062b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:39.385216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8662a94-1b16-4d6c-8b34-b747aaf9062b admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:39.385511 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8662a94-1b16-4d6c-8b34-b747aaf9062b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:39.385790 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8662a94-1b16-4d6c-8b34-b747aaf9062b admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:39.390537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8662a94-1b16-4d6c-8b34-b747aaf9062b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:39.390537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8662a94-1b16-4d6c-8b34-b747aaf9062b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:39.411245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1007/2009] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:39.411354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489af8a-137a-47da-91e8-49c5287deb10 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:39.424783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489af8a-137a-47da-91e8-49c5287deb10 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:39.429816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61973783-1d66-4ea3-abb6-8d03f3b90a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:39.430504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61973783-1d66-4ea3-abb6-8d03f3b90a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:39.430792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61973783-1d66-4ea3-abb6-8d03f3b90a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:39.431131 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61973783-1d66-4ea3-abb6-8d03f3b90a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:39.431843 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61973783-1d66-4ea3-abb6-8d03f3b90a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:39.433587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61973783-1d66-4ea3-abb6-8d03f3b90a6b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:39.449494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1003/2010] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:39.453438 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8662a94-1b16-4d6c-8b34-b747aaf9062b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:39.464173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8662a94-1b16-4d6c-8b34-b747aaf9062b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:39.469434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c304f6-03bf-4a48-afa9-456291e224ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:39.470351 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c304f6-03bf-4a48-afa9-456291e224ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:39.470626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c304f6-03bf-4a48-afa9-456291e224ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:39.470939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c304f6-03bf-4a48-afa9-456291e224ab admin admin] PATH_INFO: `/v3/projects/5ac9864959e3406591785f4f25d66bf9/users/4136101ab5f74b96987ff862b070e0ae/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:39.487551 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c304f6-03bf-4a48-afa9-456291e224ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac9864959e3406591785f4f25d66bf9, user_id=4136101ab5f74b96987ff862b070e0ae, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:39.490350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c304f6-03bf-4a48-afa9-456291e224ab admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:39.582262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1008/2011] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:39 2022] PUT /identity/v3/projects/5ac9864959e3406591785f4f25d66bf9/users/4136101ab5f74b96987ff862b070e0ae/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:39.593571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584c447-6a55-43a0-ac1b-58d0802a5cbd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:39.593845 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584c447-6a55-43a0-ac1b-58d0802a5cbd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:39.594126 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584c447-6a55-43a0-ac1b-58d0802a5cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:39.621469 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c584c447-6a55-43a0-ac1b-58d0802a5cbd None None] MFA Rules not processed for user `4136101ab5f74b96987ff862b070e0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:39.622675 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c584c447-6a55-43a0-ac1b-58d0802a5cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:39.693403 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1004/2012] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:39.698704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c304f6-03bf-4a48-afa9-456291e224ab admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:39.703662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:39.771958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c304f6-03bf-4a48-afa9-456291e224ab admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:39.777167 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd710d48-9d8d-4c53-a347-65ecb9357c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:39.777910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd710d48-9d8d-4c53-a347-65ecb9357c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:39.778101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd710d48-9d8d-4c53-a347-65ecb9357c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:39.778380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd710d48-9d8d-4c53-a347-65ecb9357c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:39.779113 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd710d48-9d8d-4c53-a347-65ecb9357c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:39.781313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd710d48-9d8d-4c53-a347-65ecb9357c8d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:39.920914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1009/2013] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:39 2022] POST /identity/v3/projects => generated 387 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:39.922373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584c447-6a55-43a0-ac1b-58d0802a5cbd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:39.933602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584c447-6a55-43a0-ac1b-58d0802a5cbd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:39.992862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61ae06-6fb0-45b8-9344-116ff2be0f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:39.993472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61ae06-6fb0-45b8-9344-116ff2be0f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:39.993753 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61ae06-6fb0-45b8-9344-116ff2be0f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:39.994026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61ae06-6fb0-45b8-9344-116ff2be0f7f admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:39.994716 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61ae06-6fb0-45b8-9344-116ff2be0f7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:39.996658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61ae06-6fb0-45b8-9344-116ff2be0f7f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:40.102872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1005/2014] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:39 2022] POST /identity/v3/users => generated 300 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:40.104306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd710d48-9d8d-4c53-a347-65ecb9357c8d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:40.118589 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd710d48-9d8d-4c53-a347-65ecb9357c8d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:40.124562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b8d42-babc-403c-bbcf-d75ecb390c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:40.125989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b8d42-babc-403c-bbcf-d75ecb390c35 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:40.126338 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b8d42-babc-403c-bbcf-d75ecb390c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:40.126959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b8d42-babc-403c-bbcf-d75ecb390c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:40.128093 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b8d42-babc-403c-bbcf-d75ecb390c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:40.130778 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b8d42-babc-403c-bbcf-d75ecb390c35 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:40.147101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1010/2015] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:40.149929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61ae06-6fb0-45b8-9344-116ff2be0f7f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:40.169996 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61ae06-6fb0-45b8-9344-116ff2be0f7f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:40.178884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5181bc-ccdc-4ada-87ae-a8c86f20ae13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:40.179851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5181bc-ccdc-4ada-87ae-a8c86f20ae13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:40.189116 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5181bc-ccdc-4ada-87ae-a8c86f20ae13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:40.189116 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5181bc-ccdc-4ada-87ae-a8c86f20ae13 admin admin] PATH_INFO: `/v3/projects/ed6e5b938daa4e478ef9d1ed093efbe7/users/ea6ac4c675ad4245b25f1a1b873712de/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:40.203169 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:40.207892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5181bc-ccdc-4ada-87ae-a8c86f20ae13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed6e5b938daa4e478ef9d1ed093efbe7, user_id=ea6ac4c675ad4245b25f1a1b873712de, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:40.210611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5181bc-ccdc-4ada-87ae-a8c86f20ae13 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:40.287328 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1006/2016] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:40 2022] PUT /identity/v3/projects/ed6e5b938daa4e478ef9d1ed093efbe7/users/ea6ac4c675ad4245b25f1a1b873712de/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:40.292056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b8d42-babc-403c-bbcf-d75ecb390c35 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:40.303728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b8d42-babc-403c-bbcf-d75ecb390c35 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:40.369956 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f47c8f-594d-4c17-bfd6-51761907aa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:40.370662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f47c8f-594d-4c17-bfd6-51761907aa6c admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:40.370949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f47c8f-594d-4c17-bfd6-51761907aa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:40.371259 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f47c8f-594d-4c17-bfd6-51761907aa6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:40.372011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f47c8f-594d-4c17-bfd6-51761907aa6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:40.373686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f47c8f-594d-4c17-bfd6-51761907aa6c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:40.383407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1011/2017] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:40.387933 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5181bc-ccdc-4ada-87ae-a8c86f20ae13 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:40.399218 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5181bc-ccdc-4ada-87ae-a8c86f20ae13 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:40.403967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61984ae2-4ba6-4a37-a112-a66449958408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:40.404857 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61984ae2-4ba6-4a37-a112-a66449958408 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:40.405138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61984ae2-4ba6-4a37-a112-a66449958408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:40.405667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61984ae2-4ba6-4a37-a112-a66449958408 admin admin] PATH_INFO: `/v3/projects/ed6e5b938daa4e478ef9d1ed093efbe7/users/ea6ac4c675ad4245b25f1a1b873712de/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:40.407699 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61984ae2-4ba6-4a37-a112-a66449958408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed6e5b938daa4e478ef9d1ed093efbe7, user_id=ea6ac4c675ad4245b25f1a1b873712de, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:40.410966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61984ae2-4ba6-4a37-a112-a66449958408 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:40.421891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1007/2018] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:40 2022] PUT /identity/v3/projects/ed6e5b938daa4e478ef9d1ed093efbe7/users/ea6ac4c675ad4245b25f1a1b873712de/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:40.433085 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b55aa-9225-4b9c-9884-52c5b3dae4ac None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:40.433376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b55aa-9225-4b9c-9884-52c5b3dae4ac None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:40.433656 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b55aa-9225-4b9c-9884-52c5b3dae4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:40.461819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bb9b55aa-9225-4b9c-9884-52c5b3dae4ac None None] MFA Rules not processed for user `ea6ac4c675ad4245b25f1a1b873712de`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:40.463045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bb9b55aa-9225-4b9c-9884-52c5b3dae4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:40.463645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61984ae2-4ba6-4a37-a112-a66449958408 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:40.536937 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61984ae2-4ba6-4a37-a112-a66449958408 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:40.541915 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43c365-a126-4066-93b0-18a3dff4aeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:40.542566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:40.542832 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43c365-a126-4066-93b0-18a3dff4aeb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:40.543114 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43c365-a126-4066-93b0-18a3dff4aeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:40.543505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43c365-a126-4066-93b0-18a3dff4aeb7 admin admin] PATH_INFO: `/v3/users/bf91706eba964fbe94c9817c43ef8bdd` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:40.544656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43c365-a126-4066-93b0-18a3dff4aeb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf91706eba964fbe94c9817c43ef8bdd)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:40.546567 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43c365-a126-4066-93b0-18a3dff4aeb7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:40.548647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1012/2019] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:40.560478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b55aa-9225-4b9c-9884-52c5b3dae4ac None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:40.618909 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b55aa-9225-4b9c-9884-52c5b3dae4ac None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:40.622666 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cf8cc-8b42-47a7-b632-c7798aba230b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:40.623366 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cf8cc-8b42-47a7-b632-c7798aba230b service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:40.623661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cf8cc-8b42-47a7-b632-c7798aba230b service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:40.623938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cf8cc-8b42-47a7-b632-c7798aba230b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:40.634230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cf8cc-8b42-47a7-b632-c7798aba230b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:40.635753 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cf8cc-8b42-47a7-b632-c7798aba230b service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:40.646498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1013/2020] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:40.767546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b43c365-a126-4066-93b0-18a3dff4aeb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf91706eba964fbe94c9817c43ef8bdd'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:40.783038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b43c365-a126-4066-93b0-18a3dff4aeb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf91706eba964fbe94c9817c43ef8bdd'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:40.783866 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b43c365-a126-4066-93b0-18a3dff4aeb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf91706eba964fbe94c9817c43ef8bdd'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:40.793148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b43c365-a126-4066-93b0-18a3dff4aeb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf91706eba964fbe94c9817c43ef8bdd'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:40.793930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b43c365-a126-4066-93b0-18a3dff4aeb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf91706eba964fbe94c9817c43ef8bdd'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:40.872429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1008/2021] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:40 2022] DELETE /identity/v3/users/bf91706eba964fbe94c9817c43ef8bdd => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:40.876918 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c64fd-48d3-49ca-b9e6-d66d4b13d1a2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:40.878034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c64fd-48d3-49ca-b9e6-d66d4b13d1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:40.878034 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c64fd-48d3-49ca-b9e6-d66d4b13d1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:40.897494 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1b4c64fd-48d3-49ca-b9e6-d66d4b13d1a2 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:40.899130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1b4c64fd-48d3-49ca-b9e6-d66d4b13d1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:40.961002 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1014/2022] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:40.970545 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43c365-a126-4066-93b0-18a3dff4aeb7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:40.972348 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1b43c365-a126-4066-93b0-18a3dff4aeb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:41.034764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43c365-a126-4066-93b0-18a3dff4aeb7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:41.038735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cd7da-dfd6-425b-ba03-f44b50de2771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:41.039643 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cd7da-dfd6-425b-ba03-f44b50de2771 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:41.039928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cd7da-dfd6-425b-ba03-f44b50de2771 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:41.040201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cd7da-dfd6-425b-ba03-f44b50de2771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:41.050594 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cd7da-dfd6-425b-ba03-f44b50de2771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:41.051983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cd7da-dfd6-425b-ba03-f44b50de2771 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:41.055863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:41.062106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1009/2023] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:41.527306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c64fd-48d3-49ca-b9e6-d66d4b13d1a2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:41.531233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1b4c64fd-48d3-49ca-b9e6-d66d4b13d1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:41.542613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c64fd-48d3-49ca-b9e6-d66d4b13d1a2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:41.547923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a5e7e-c774-407e-8468-ce791f691752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:41.548906 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a5e7e-c774-407e-8468-ce791f691752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:41.549192 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a5e7e-c774-407e-8468-ce791f691752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:41.549526 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a5e7e-c774-407e-8468-ce791f691752 admin admin] PATH_INFO: `/v3/projects/d02aeaea285746ea959615c310785575` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:41.550793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a5e7e-c774-407e-8468-ce791f691752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d02aeaea285746ea959615c310785575)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:41.552702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a5e7e-c774-407e-8468-ce791f691752 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:41.678357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a81a5e7e-c774-407e-8468-ce791f691752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd02aeaea285746ea959615c310785575', 'request_id': 'req-a81a5e7e-c774-407e-8468-ce791f691752'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:41.686224 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a81a5e7e-c774-407e-8468-ce791f691752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd02aeaea285746ea959615c310785575', 'request_id': 'req-a81a5e7e-c774-407e-8468-ce791f691752'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:41.773894 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cd7da-dfd6-425b-ba03-f44b50de2771 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:41.775312 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7a6cd7da-dfd6-425b-ba03-f44b50de2771 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:41.786012 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a81a5e7e-c774-407e-8468-ce791f691752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd02aeaea285746ea959615c310785575', 'request_id': 'req-a81a5e7e-c774-407e-8468-ce791f691752'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:41.799865 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a81a5e7e-c774-407e-8468-ce791f691752 admin admin] The token cache is being invalidate because project d02aeaea285746ea959615c310785575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:41.800509 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a81a5e7e-c774-407e-8468-ce791f691752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:41.801188 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a81a5e7e-c774-407e-8468-ce791f691752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:41.803159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1015/2024] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:41 2022] DELETE /identity/v3/projects/d02aeaea285746ea959615c310785575 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:41.806843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a5e7e-c774-407e-8468-ce791f691752 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:41.808506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a81a5e7e-c774-407e-8468-ce791f691752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:41.849889 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:41.860987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cd7da-dfd6-425b-ba03-f44b50de2771 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:41.903014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a5e7e-c774-407e-8468-ce791f691752 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:41.909748 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e19a39-a074-4592-8db7-96ed2f835a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:41.911252 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e19a39-a074-4592-8db7-96ed2f835a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:41.916006 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e19a39-a074-4592-8db7-96ed2f835a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:41.916006 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e19a39-a074-4592-8db7-96ed2f835a00 admin admin] PATH_INFO: `/v3/users/3fa540ae0ebf4290a334fed806ff7d88` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:41.916006 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e19a39-a074-4592-8db7-96ed2f835a00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fa540ae0ebf4290a334fed806ff7d88)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:41.916249 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e19a39-a074-4592-8db7-96ed2f835a00 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:41.918007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3ae94-a13f-4089-8495-2aed4dd28aad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:41.918891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3ae94-a13f-4089-8495-2aed4dd28aad service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:41.919454 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3ae94-a13f-4089-8495-2aed4dd28aad service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:41.919832 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3ae94-a13f-4089-8495-2aed4dd28aad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:41.921852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-abf3ae94-a13f-4089-8495-2aed4dd28aad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:41.985816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3ae94-a13f-4089-8495-2aed4dd28aad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:41.988003 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3ae94-a13f-4089-8495-2aed4dd28aad service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:41.997778 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1010/2025] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:19:42.081674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-33e19a39-a074-4592-8db7-96ed2f835a00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fa540ae0ebf4290a334fed806ff7d88'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:42.100386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-33e19a39-a074-4592-8db7-96ed2f835a00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fa540ae0ebf4290a334fed806ff7d88'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:42.278714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-33e19a39-a074-4592-8db7-96ed2f835a00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fa540ae0ebf4290a334fed806ff7d88'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:42.281115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-33e19a39-a074-4592-8db7-96ed2f835a00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fa540ae0ebf4290a334fed806ff7d88'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:42.297175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-33e19a39-a074-4592-8db7-96ed2f835a00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fa540ae0ebf4290a334fed806ff7d88'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:42.304239 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1016/2026] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:41 2022] DELETE /identity/v3/users/3fa540ae0ebf4290a334fed806ff7d88 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:42.732120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3ae94-a13f-4089-8495-2aed4dd28aad service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:42.745742 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3ae94-a13f-4089-8495-2aed4dd28aad service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:42.815518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425b0f1-0f47-43ee-8a7c-1c9d2247d476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:42.826058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425b0f1-0f47-43ee-8a7c-1c9d2247d476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:42.826367 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425b0f1-0f47-43ee-8a7c-1c9d2247d476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:42.826656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425b0f1-0f47-43ee-8a7c-1c9d2247d476 admin admin] PATH_INFO: `/v3/projects/b286849fbbff4d71b238ebad21903abc` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:42.834512 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425b0f1-0f47-43ee-8a7c-1c9d2247d476 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b286849fbbff4d71b238ebad21903abc)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:42.836841 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425b0f1-0f47-43ee-8a7c-1c9d2247d476 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:42.853171 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:42.991552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e425b0f1-0f47-43ee-8a7c-1c9d2247d476 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b286849fbbff4d71b238ebad21903abc', 'request_id': 'req-e425b0f1-0f47-43ee-8a7c-1c9d2247d476'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:43.001225 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e425b0f1-0f47-43ee-8a7c-1c9d2247d476 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b286849fbbff4d71b238ebad21903abc', 'request_id': 'req-e425b0f1-0f47-43ee-8a7c-1c9d2247d476'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:43.113183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e425b0f1-0f47-43ee-8a7c-1c9d2247d476 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b286849fbbff4d71b238ebad21903abc', 'request_id': 'req-e425b0f1-0f47-43ee-8a7c-1c9d2247d476'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:43.124298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e425b0f1-0f47-43ee-8a7c-1c9d2247d476 admin admin] The token cache is being invalidate because project b286849fbbff4d71b238ebad21903abc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:43.124625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e425b0f1-0f47-43ee-8a7c-1c9d2247d476 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:43.125477 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e425b0f1-0f47-43ee-8a7c-1c9d2247d476 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:43.127494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1011/2027] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:42 2022] DELETE /identity/v3/projects/b286849fbbff4d71b238ebad21903abc => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:43.132676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe9484-4b0a-4777-9bae-fc615b7db9ec None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:43.133195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe9484-4b0a-4777-9bae-fc615b7db9ec None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:43.133362 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe9484-4b0a-4777-9bae-fc615b7db9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:43.152260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-15fe9484-4b0a-4777-9bae-fc615b7db9ec None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:43.153365 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-15fe9484-4b0a-4777-9bae-fc615b7db9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:43.214255 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1017/2028] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:43.222754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7460c57-09f5-4472-9fad-0d07911c7330 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:43.223049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7460c57-09f5-4472-9fad-0d07911c7330 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:43.223370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7460c57-09f5-4472-9fad-0d07911c7330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:43.242341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c7460c57-09f5-4472-9fad-0d07911c7330 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:43.248360 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c7460c57-09f5-4472-9fad-0d07911c7330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:43.251649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1012/2029] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:43.256051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe9484-4b0a-4777-9bae-fc615b7db9ec None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:43.265722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe9484-4b0a-4777-9bae-fc615b7db9ec None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:43.271422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e4695-0d99-4ca8-a30b-c6b054fc12f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:43.271422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e4695-0d99-4ca8-a30b-c6b054fc12f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:43.271422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e4695-0d99-4ca8-a30b-c6b054fc12f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:43.271766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e4695-0d99-4ca8-a30b-c6b054fc12f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:43.272204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e4695-0d99-4ca8-a30b-c6b054fc12f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:19:43.272516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e4695-0d99-4ca8-a30b-c6b054fc12f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:43.274103 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e4695-0d99-4ca8-a30b-c6b054fc12f0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:43.286521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1018/2030] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:19:43.301821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7460c57-09f5-4472-9fad-0d07911c7330 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:43.311151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7460c57-09f5-4472-9fad-0d07911c7330 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:43.316203 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f44f01-eec1-4798-8d31-14fae4a48ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:43.316810 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f44f01-eec1-4798-8d31-14fae4a48ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:43.317073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f44f01-eec1-4798-8d31-14fae4a48ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:43.317344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f44f01-eec1-4798-8d31-14fae4a48ae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:43.318073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f44f01-eec1-4798-8d31-14fae4a48ae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:43.319960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f44f01-eec1-4798-8d31-14fae4a48ae1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:43.404964 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1013/2031] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:43 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:43.409991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e4695-0d99-4ca8-a30b-c6b054fc12f0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:43.419947 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e4695-0d99-4ca8-a30b-c6b054fc12f0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:43.450212 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:43.479572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f64e69-ebcb-43d3-9c9b-490780ba3bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:43.480245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f64e69-ebcb-43d3-9c9b-490780ba3bdd admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:43.480530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f64e69-ebcb-43d3-9c9b-490780ba3bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:43.480805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f64e69-ebcb-43d3-9c9b-490780ba3bdd admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:43.481493 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f64e69-ebcb-43d3-9c9b-490780ba3bdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:43.483440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f64e69-ebcb-43d3-9c9b-490780ba3bdd admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:43.570245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1019/2032] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:43 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:43.574398 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f44f01-eec1-4798-8d31-14fae4a48ae1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:43.585693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f44f01-eec1-4798-8d31-14fae4a48ae1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:43.590386 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee2ee6-212a-4cb2-b6fe-cab4b10c04e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:43.591052 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee2ee6-212a-4cb2-b6fe-cab4b10c04e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:43.591380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee2ee6-212a-4cb2-b6fe-cab4b10c04e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:43.591663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee2ee6-212a-4cb2-b6fe-cab4b10c04e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:43.592384 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ee2ee6-212a-4cb2-b6fe-cab4b10c04e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:43.593982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ee2ee6-212a-4cb2-b6fe-cab4b10c04e3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:43.605302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1014/2033] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:43.609246 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f64e69-ebcb-43d3-9c9b-490780ba3bdd admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:43.619103 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f64e69-ebcb-43d3-9c9b-490780ba3bdd admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:43.623952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f74f6-a4fd-46a4-8828-4c7524840be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:43.626649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f74f6-a4fd-46a4-8828-4c7524840be4 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:43.626649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f74f6-a4fd-46a4-8828-4c7524840be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:43.626649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f74f6-a4fd-46a4-8828-4c7524840be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:43.626649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66f74f6-a4fd-46a4-8828-4c7524840be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:43.627598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66f74f6-a4fd-46a4-8828-4c7524840be4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:43.637401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1020/2034] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:43.641586 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee2ee6-212a-4cb2-b6fe-cab4b10c04e3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:43.652537 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee2ee6-212a-4cb2-b6fe-cab4b10c04e3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:43.657549 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2fdfa-aa3e-458d-a291-8263f8956c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:43.658994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2fdfa-aa3e-458d-a291-8263f8956c23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:43.659316 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2fdfa-aa3e-458d-a291-8263f8956c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:43.659596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2fdfa-aa3e-458d-a291-8263f8956c23 admin admin] PATH_INFO: `/v3/projects/b0fa019dc14d4e40a77d6b7db5cad9be/users/c95b2f1348424e6fa05c9599d4e3611a/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:43.680106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2fdfa-aa3e-458d-a291-8263f8956c23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0fa019dc14d4e40a77d6b7db5cad9be, user_id=c95b2f1348424e6fa05c9599d4e3611a, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:43.683976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2fdfa-aa3e-458d-a291-8263f8956c23 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:43.767305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1015/2035] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:43 2022] PUT /identity/v3/projects/b0fa019dc14d4e40a77d6b7db5cad9be/users/c95b2f1348424e6fa05c9599d4e3611a/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:43.773554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a8993-86c0-417b-b5dc-b53bf06410ad None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:43.773865 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a8993-86c0-417b-b5dc-b53bf06410ad None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:43.774152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a8993-86c0-417b-b5dc-b53bf06410ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:43.793185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-065a8993-86c0-417b-b5dc-b53bf06410ad None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:43.794401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-065a8993-86c0-417b-b5dc-b53bf06410ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:43.888694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1021/2036] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:43.897779 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2fdfa-aa3e-458d-a291-8263f8956c23 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:43.901445 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e3c2fdfa-aa3e-458d-a291-8263f8956c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:43.985292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2fdfa-aa3e-458d-a291-8263f8956c23 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:43.995440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da34ad3-d073-48cb-b07c-91808673f120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:43.995440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da34ad3-d073-48cb-b07c-91808673f120 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:43.995440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da34ad3-d073-48cb-b07c-91808673f120 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:43.995440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da34ad3-d073-48cb-b07c-91808673f120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:44.007770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da34ad3-d073-48cb-b07c-91808673f120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:44.009874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da34ad3-d073-48cb-b07c-91808673f120 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:44.027284 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1016/2037] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:45.497179 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0873694-b7a7-4670-afcd-86a49b09ca26 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:45.498270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0873694-b7a7-4670-afcd-86a49b09ca26 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:45.498716 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0873694-b7a7-4670-afcd-86a49b09ca26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:45.532839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b0873694-b7a7-4670-afcd-86a49b09ca26 None None] MFA Rules not processed for user `c95b2f1348424e6fa05c9599d4e3611a`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:45.534976 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b0873694-b7a7-4670-afcd-86a49b09ca26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:45.625342 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1022/2038] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:45.637139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da34ad3-d073-48cb-b07c-91808673f120 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:45.638684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0da34ad3-d073-48cb-b07c-91808673f120 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:45.698124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:45.714462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da34ad3-d073-48cb-b07c-91808673f120 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:45.718240 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692d746-0fbb-422e-aebc-070b32e86173 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:45.718910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692d746-0fbb-422e-aebc-070b32e86173 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:45.719226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692d746-0fbb-422e-aebc-070b32e86173 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:45.719515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692d746-0fbb-422e-aebc-070b32e86173 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:45.729058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692d746-0fbb-422e-aebc-070b32e86173 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:45.730499 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692d746-0fbb-422e-aebc-070b32e86173 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:45.741078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1017/2039] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:46.101167 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0873694-b7a7-4670-afcd-86a49b09ca26 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:46.105999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b0873694-b7a7-4670-afcd-86a49b09ca26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:46.125387 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0873694-b7a7-4670-afcd-86a49b09ca26 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:46.131669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df9867-bd38-4bb5-aeef-48545d4182e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:46.135641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df9867-bd38-4bb5-aeef-48545d4182e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:46.136759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df9867-bd38-4bb5-aeef-48545d4182e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:46.137168 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df9867-bd38-4bb5-aeef-48545d4182e9 admin admin] PATH_INFO: `/v3/users/e1442c910c774b308533174792e31a84` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:46.138958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df9867-bd38-4bb5-aeef-48545d4182e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1442c910c774b308533174792e31a84)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:46.141470 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df9867-bd38-4bb5-aeef-48545d4182e9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:46.374967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:46.385091 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-13df9867-bd38-4bb5-aeef-48545d4182e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1442c910c774b308533174792e31a84'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:46.397874 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-13df9867-bd38-4bb5-aeef-48545d4182e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1442c910c774b308533174792e31a84'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:46.479101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-13df9867-bd38-4bb5-aeef-48545d4182e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1442c910c774b308533174792e31a84'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:46.484657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-13df9867-bd38-4bb5-aeef-48545d4182e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1442c910c774b308533174792e31a84'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:46.488534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-13df9867-bd38-4bb5-aeef-48545d4182e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1442c910c774b308533174792e31a84'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:46.495537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1023/2040] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:46 2022] DELETE /identity/v3/users/e1442c910c774b308533174792e31a84 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:46.499802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84227ff0-7397-4b06-b2af-39bd60a591b5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:46.501719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84227ff0-7397-4b06-b2af-39bd60a591b5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:46.501719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84227ff0-7397-4b06-b2af-39bd60a591b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:46.526793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-84227ff0-7397-4b06-b2af-39bd60a591b5 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:46.528217 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-84227ff0-7397-4b06-b2af-39bd60a591b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:46.600661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1018/2041] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:46.610392 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df9867-bd38-4bb5-aeef-48545d4182e9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:46.612856 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-13df9867-bd38-4bb5-aeef-48545d4182e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:46.685507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df9867-bd38-4bb5-aeef-48545d4182e9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:46.701677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff5eca-6c73-45fb-af90-4a9d89de83e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:46.703760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff5eca-6c73-45fb-af90-4a9d89de83e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:46.704129 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff5eca-6c73-45fb-af90-4a9d89de83e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:46.704673 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff5eca-6c73-45fb-af90-4a9d89de83e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:46.720627 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff5eca-6c73-45fb-af90-4a9d89de83e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:46.723675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff5eca-6c73-45fb-af90-4a9d89de83e8 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:46.735965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1024/2042] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:47.191049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84227ff0-7397-4b06-b2af-39bd60a591b5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:47.192742 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-84227ff0-7397-4b06-b2af-39bd60a591b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:47.207657 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84227ff0-7397-4b06-b2af-39bd60a591b5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:47.215461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd3d79-7584-4935-b528-ea379e87ca17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:47.215461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd3d79-7584-4935-b528-ea379e87ca17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:47.215461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd3d79-7584-4935-b528-ea379e87ca17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:47.215461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd3d79-7584-4935-b528-ea379e87ca17 admin admin] PATH_INFO: `/v3/projects/5a3257f43c7c4c19a3a74209b64aa27f` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:47.216324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd3d79-7584-4935-b528-ea379e87ca17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a3257f43c7c4c19a3a74209b64aa27f)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:47.218423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd3d79-7584-4935-b528-ea379e87ca17 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:47.259866 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff5eca-6c73-45fb-af90-4a9d89de83e8 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:47.261303 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-63ff5eca-6c73-45fb-af90-4a9d89de83e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:47.324869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff5eca-6c73-45fb-af90-4a9d89de83e8 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:47.337420 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-06dd3d79-7584-4935-b528-ea379e87ca17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a3257f43c7c4c19a3a74209b64aa27f', 'request_id': 'req-06dd3d79-7584-4935-b528-ea379e87ca17'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:47.344704 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-06dd3d79-7584-4935-b528-ea379e87ca17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a3257f43c7c4c19a3a74209b64aa27f', 'request_id': 'req-06dd3d79-7584-4935-b528-ea379e87ca17'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:47.374887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d907493-d700-4f72-8dcb-cb05577f4446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:47.375573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d907493-d700-4f72-8dcb-cb05577f4446 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:47.375865 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d907493-d700-4f72-8dcb-cb05577f4446 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:47.376178 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d907493-d700-4f72-8dcb-cb05577f4446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:47.378634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5d907493-d700-4f72-8dcb-cb05577f4446 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:47.404618 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:47.408719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-06dd3d79-7584-4935-b528-ea379e87ca17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a3257f43c7c4c19a3a74209b64aa27f', 'request_id': 'req-06dd3d79-7584-4935-b528-ea379e87ca17'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:47.421571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-06dd3d79-7584-4935-b528-ea379e87ca17 admin admin] The token cache is being invalidate because project 5a3257f43c7c4c19a3a74209b64aa27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:47.422152 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-06dd3d79-7584-4935-b528-ea379e87ca17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:47.423220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-06dd3d79-7584-4935-b528-ea379e87ca17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:47.425951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1019/2043] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:47 2022] DELETE /identity/v3/projects/5a3257f43c7c4c19a3a74209b64aa27f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:47.430855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e3c7d-6532-4952-8bda-29a00d8f2402 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:47.431142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e3c7d-6532-4952-8bda-29a00d8f2402 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:47.431560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e3c7d-6532-4952-8bda-29a00d8f2402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:47.443142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d907493-d700-4f72-8dcb-cb05577f4446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:47.444680 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d907493-d700-4f72-8dcb-cb05577f4446 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:47.451663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-004e3c7d-6532-4952-8bda-29a00d8f2402 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:47.452870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-004e3c7d-6532-4952-8bda-29a00d8f2402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:47.454041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1025/2044] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:19:47.527094 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1020/2045] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:47.537328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be55433-e2e1-41f1-b9df-2d4bee98c520 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:47.540616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be55433-e2e1-41f1-b9df-2d4bee98c520 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:47.540616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be55433-e2e1-41f1-b9df-2d4bee98c520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:47.579020 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0be55433-e2e1-41f1-b9df-2d4bee98c520 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:47.580519 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0be55433-e2e1-41f1-b9df-2d4bee98c520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:47.592414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1026/2046] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:47.597203 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e3c7d-6532-4952-8bda-29a00d8f2402 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:47.603535 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:47.607916 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e3c7d-6532-4952-8bda-29a00d8f2402 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:47.614403 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebf182-a147-4caa-9b99-3eaf48f7fd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:47.619347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebf182-a147-4caa-9b99-3eaf48f7fd5a admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:47.619347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebf182-a147-4caa-9b99-3eaf48f7fd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:47.619347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebf182-a147-4caa-9b99-3eaf48f7fd5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:47.619347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebf182-a147-4caa-9b99-3eaf48f7fd5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:19:47.619347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebf182-a147-4caa-9b99-3eaf48f7fd5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:47.619347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebf182-a147-4caa-9b99-3eaf48f7fd5a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:47.638804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1021/2047] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:19:47.639935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be55433-e2e1-41f1-b9df-2d4bee98c520 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:47.649378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be55433-e2e1-41f1-b9df-2d4bee98c520 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:47.654073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674c790-384e-4900-b1db-b18b0f46fbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:47.655430 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674c790-384e-4900-b1db-b18b0f46fbd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:47.655430 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674c790-384e-4900-b1db-b18b0f46fbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:47.655645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674c790-384e-4900-b1db-b18b0f46fbd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:47.656337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674c790-384e-4900-b1db-b18b0f46fbd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:47.658940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674c790-384e-4900-b1db-b18b0f46fbd8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:47.790054 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63986fb-8d78-4176-b6ac-27700b778d8f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:47.790343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63986fb-8d78-4176-b6ac-27700b778d8f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:47.790619 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63986fb-8d78-4176-b6ac-27700b778d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:47.793889 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1027/2048] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:47 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:47.797488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674c790-384e-4900-b1db-b18b0f46fbd8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:47.810742 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b63986fb-8d78-4176-b6ac-27700b778d8f None None] MFA Rules not processed for user `c95b2f1348424e6fa05c9599d4e3611a`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:47.812415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b63986fb-8d78-4176-b6ac-27700b778d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:47.865954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674c790-384e-4900-b1db-b18b0f46fbd8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:47.871902 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651070b-5ce5-4b73-ad1b-d900f486a3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:47.872653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651070b-5ce5-4b73-ad1b-d900f486a3ab admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:47.872944 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651070b-5ce5-4b73-ad1b-d900f486a3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:47.873223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651070b-5ce5-4b73-ad1b-d900f486a3ab admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:47.873931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651070b-5ce5-4b73-ad1b-d900f486a3ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:47.876043 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651070b-5ce5-4b73-ad1b-d900f486a3ab admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:47.880591 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1022/2049] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:47.970115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1028/2050] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:47 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:47.975755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63986fb-8d78-4176-b6ac-27700b778d8f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:47.990763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63986fb-8d78-4176-b6ac-27700b778d8f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:47.996271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434316b-907c-4c25-bb7d-f8157d21b12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:47.997041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434316b-907c-4c25-bb7d-f8157d21b12e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:47.997338 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434316b-907c-4c25-bb7d-f8157d21b12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:47.997602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434316b-907c-4c25-bb7d-f8157d21b12e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:47.998333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434316b-907c-4c25-bb7d-f8157d21b12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:48.000485 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434316b-907c-4c25-bb7d-f8157d21b12e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:48.015226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1023/2051] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:48.016908 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651070b-5ce5-4b73-ad1b-d900f486a3ab admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:48.029984 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651070b-5ce5-4b73-ad1b-d900f486a3ab admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:48.043440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e65705-d30f-4257-b469-097f7dbffcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:48.044195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e65705-d30f-4257-b469-097f7dbffcc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:48.044519 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e65705-d30f-4257-b469-097f7dbffcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:48.044796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e65705-d30f-4257-b469-097f7dbffcc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:48.045775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e65705-d30f-4257-b469-097f7dbffcc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:48.047470 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e65705-d30f-4257-b469-097f7dbffcc3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:48.057808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1029/2052] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:48.062326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434316b-907c-4c25-bb7d-f8157d21b12e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:48.073175 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434316b-907c-4c25-bb7d-f8157d21b12e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:48.078785 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b150ded-b404-4b23-a39f-721a24f5962c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:48.079641 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b150ded-b404-4b23-a39f-721a24f5962c admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:48.080559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b150ded-b404-4b23-a39f-721a24f5962c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:48.080955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b150ded-b404-4b23-a39f-721a24f5962c admin admin] PATH_INFO: `/v3/projects/47f832eeb1fd4414b11e5ebb190b3afa/users/bfaf9576d8af4a34aef4e6ecd54d8eb4/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:48.098834 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b150ded-b404-4b23-a39f-721a24f5962c admin admin] RBAC: Authorizing `identity:create_grant(project_id=47f832eeb1fd4414b11e5ebb190b3afa, user_id=bfaf9576d8af4a34aef4e6ecd54d8eb4, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:48.103792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b150ded-b404-4b23-a39f-721a24f5962c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:48.223479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1024/2053] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:48 2022] PUT /identity/v3/projects/47f832eeb1fd4414b11e5ebb190b3afa/users/bfaf9576d8af4a34aef4e6ecd54d8eb4/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:48.234213 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728d4df-7173-4275-b72b-4fdfe4f20475 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:48.234500 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728d4df-7173-4275-b72b-4fdfe4f20475 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:48.234801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728d4df-7173-4275-b72b-4fdfe4f20475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:48.269190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7728d4df-7173-4275-b72b-4fdfe4f20475 None None] MFA Rules not processed for user `bfaf9576d8af4a34aef4e6ecd54d8eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:48.270538 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7728d4df-7173-4275-b72b-4fdfe4f20475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:48.348493 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1030/2054] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:48.353161 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b150ded-b404-4b23-a39f-721a24f5962c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:48.391476 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb799cb5-f6c1-4486-af8f-277f86f06907 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:48.391854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb799cb5-f6c1-4486-af8f-277f86f06907 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:48.392245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb799cb5-f6c1-4486-af8f-277f86f06907 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:48.394431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1031/2055] 10.222.0.67 () {54 vars in 912 bytes} [Fri Jun 24 15:19:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:19:48.399564 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cfc36-db5d-463f-a2a1-3fbf1c9fd639 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:48.399859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cfc36-db5d-463f-a2a1-3fbf1c9fd639 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:48.400141 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cfc36-db5d-463f-a2a1-3fbf1c9fd639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:48.407851 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:48.418016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b150ded-b404-4b23-a39f-721a24f5962c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:48.419482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5a8cfc36-db5d-463f-a2a1-3fbf1c9fd639 None None] MFA Rules not processed for user `95e7fb8f48dd4348b54e3694a8410bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:48.421325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5a8cfc36-db5d-463f-a2a1-3fbf1c9fd639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:48.422809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83b21e-9159-4aab-b018-7d3648ace106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:48.423495 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83b21e-9159-4aab-b018-7d3648ace106 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:48.423817 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83b21e-9159-4aab-b018-7d3648ace106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:48.424245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83b21e-9159-4aab-b018-7d3648ace106 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:48.424952 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e83b21e-9159-4aab-b018-7d3648ace106 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:48.426855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e83b21e-9159-4aab-b018-7d3648ace106 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:48.471594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1032/2056] 10.222.0.67 () {58 vars in 1009 bytes} [Fri Jun 24 15:19:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:48.482482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8cfc36-db5d-463f-a2a1-3fbf1c9fd639 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:48.483803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5a8cfc36-db5d-463f-a2a1-3fbf1c9fd639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:48.493868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8cfc36-db5d-463f-a2a1-3fbf1c9fd639 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:48.497474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0e4e6-fdbd-4b7c-9244-4fe3a3db62a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:48.498099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0e4e6-fdbd-4b7c-9244-4fe3a3db62a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:48.498387 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0e4e6-fdbd-4b7c-9244-4fe3a3db62a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:48.498669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0e4e6-fdbd-4b7c-9244-4fe3a3db62a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:48.506237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0e4e6-fdbd-4b7c-9244-4fe3a3db62a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:48.507693 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0e4e6-fdbd-4b7c-9244-4fe3a3db62a1 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:48.515342 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1033/2057] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:19:48.529253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1025/2058] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:19:48 2022] POST /identity/v3/projects => generated 371 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:19:48.530599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0e4e6-fdbd-4b7c-9244-4fe3a3db62a1 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:48.542087 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0e4e6-fdbd-4b7c-9244-4fe3a3db62a1 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:48.607314 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e72d1b-d6c1-4b64-9eff-e222bcd3f6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:48.607936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e72d1b-d6c1-4b64-9eff-e222bcd3f6be admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:48.608199 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e72d1b-d6c1-4b64-9eff-e222bcd3f6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:48.608561 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e72d1b-d6c1-4b64-9eff-e222bcd3f6be admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:48.609364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e72d1b-d6c1-4b64-9eff-e222bcd3f6be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:48.611844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e72d1b-d6c1-4b64-9eff-e222bcd3f6be admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:48.727691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1034/2059] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:19:48 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:19:48.731670 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83b21e-9159-4aab-b018-7d3648ace106 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:48.743766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83b21e-9159-4aab-b018-7d3648ace106 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:48.749223 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcaa7b-94e3-48c2-8a87-b7ce57fd98ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:48.750124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcaa7b-94e3-48c2-8a87-b7ce57fd98ac admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:48.750409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcaa7b-94e3-48c2-8a87-b7ce57fd98ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:48.750742 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcaa7b-94e3-48c2-8a87-b7ce57fd98ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:48.752502 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdcaa7b-94e3-48c2-8a87-b7ce57fd98ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:48.754320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdcaa7b-94e3-48c2-8a87-b7ce57fd98ac admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:48.766321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1026/2060] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:19:48.770899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e72d1b-d6c1-4b64-9eff-e222bcd3f6be admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:48.782578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e72d1b-d6c1-4b64-9eff-e222bcd3f6be admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:48.788304 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2a66d-b4af-4736-8b0a-d2be5a78b83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:48.789490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2a66d-b4af-4736-8b0a-d2be5a78b83a admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:48.790158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2a66d-b4af-4736-8b0a-d2be5a78b83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:48.790612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2a66d-b4af-4736-8b0a-d2be5a78b83a admin admin] PATH_INFO: `/v3/projects/ef142d225c6540829c8774eba2119508/users/ac0b94a3bfc648e7ba728b98c3a5dbde/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:48.811833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2a66d-b4af-4736-8b0a-d2be5a78b83a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef142d225c6540829c8774eba2119508, user_id=ac0b94a3bfc648e7ba728b98c3a5dbde, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:48.815801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2a66d-b4af-4736-8b0a-d2be5a78b83a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:48.875967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1035/2061] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:48 2022] PUT /identity/v3/projects/ef142d225c6540829c8774eba2119508/users/ac0b94a3bfc648e7ba728b98c3a5dbde/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:48.880771 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcaa7b-94e3-48c2-8a87-b7ce57fd98ac admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:48.892675 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcaa7b-94e3-48c2-8a87-b7ce57fd98ac admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:48.930369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:48.955999 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75cb7c-7f27-45fb-960f-f3c7a9d6602a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:48.956785 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75cb7c-7f27-45fb-960f-f3c7a9d6602a admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:48.957059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75cb7c-7f27-45fb-960f-f3c7a9d6602a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:48.957339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75cb7c-7f27-45fb-960f-f3c7a9d6602a admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:48.958059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75cb7c-7f27-45fb-960f-f3c7a9d6602a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:48.959683 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75cb7c-7f27-45fb-960f-f3c7a9d6602a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:48.970039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1027/2062] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:19:48.975316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2a66d-b4af-4736-8b0a-d2be5a78b83a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:48.990729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2a66d-b4af-4736-8b0a-d2be5a78b83a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:48.995730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c439f5e-2e3a-4fc6-948a-760d06c4710a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:48.996927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c439f5e-2e3a-4fc6-948a-760d06c4710a admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:48.997211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c439f5e-2e3a-4fc6-948a-760d06c4710a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:48.997492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c439f5e-2e3a-4fc6-948a-760d06c4710a admin admin] PATH_INFO: `/v3/projects/ef142d225c6540829c8774eba2119508/users/ac0b94a3bfc648e7ba728b98c3a5dbde/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:48.999942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c439f5e-2e3a-4fc6-948a-760d06c4710a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef142d225c6540829c8774eba2119508, user_id=ac0b94a3bfc648e7ba728b98c3a5dbde, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:49.002808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c439f5e-2e3a-4fc6-948a-760d06c4710a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:49.021391 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1036/2063] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:19:48 2022] PUT /identity/v3/projects/ef142d225c6540829c8774eba2119508/users/ac0b94a3bfc648e7ba728b98c3a5dbde/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:49.039536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202aee9-1363-41af-b79b-5ce6eaf556c9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:49.039893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202aee9-1363-41af-b79b-5ce6eaf556c9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:49.040125 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202aee9-1363-41af-b79b-5ce6eaf556c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:49.088103 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1202aee9-1363-41af-b79b-5ce6eaf556c9 None None] MFA Rules not processed for user `ac0b94a3bfc648e7ba728b98c3a5dbde`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:49.089572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1202aee9-1363-41af-b79b-5ce6eaf556c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:49.213048 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1028/2064] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:49 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:49.245849 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c439f5e-2e3a-4fc6-948a-760d06c4710a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:49.247652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1c439f5e-2e3a-4fc6-948a-760d06c4710a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:49.308745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c439f5e-2e3a-4fc6-948a-760d06c4710a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:49.312595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2a236-5e71-4c61-8593-0564e661ead2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:49.313376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2a236-5e71-4c61-8593-0564e661ead2 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:49.313664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2a236-5e71-4c61-8593-0564e661ead2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:49.313942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2a236-5e71-4c61-8593-0564e661ead2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:49.332604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:49.374049 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2a236-5e71-4c61-8593-0564e661ead2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:49.375546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2a236-5e71-4c61-8593-0564e661ead2 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:49.384780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1037/2065] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:19:56.435447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202aee9-1363-41af-b79b-5ce6eaf556c9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:56.437122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1202aee9-1363-41af-b79b-5ce6eaf556c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:56.572804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202aee9-1363-41af-b79b-5ce6eaf556c9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:56.577780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff137b-f65d-4eb4-b09b-d33a51a5aff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:56.578741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff137b-f65d-4eb4-b09b-d33a51a5aff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:56.579154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff137b-f65d-4eb4-b09b-d33a51a5aff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:56.579468 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff137b-f65d-4eb4-b09b-d33a51a5aff7 admin admin] PATH_INFO: `/v3/users/6577082bc2e6446591191a439b27de38` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:56.581371 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff137b-f65d-4eb4-b09b-d33a51a5aff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6577082bc2e6446591191a439b27de38)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:56.583330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff137b-f65d-4eb4-b09b-d33a51a5aff7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:56.803896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2a236-5e71-4c61-8593-0564e661ead2 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:56.818728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2a236-5e71-4c61-8593-0564e661ead2 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:56.823807 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f433d-5535-42b3-9101-4976b12586ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:56.824710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f433d-5535-42b3-9101-4976b12586ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:56.825080 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f433d-5535-42b3-9101-4976b12586ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:56.825487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f433d-5535-42b3-9101-4976b12586ea admin admin] PATH_INFO: `/v3/users/bfaf9576d8af4a34aef4e6ecd54d8eb4` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:56.826866 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f433d-5535-42b3-9101-4976b12586ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfaf9576d8af4a34aef4e6ecd54d8eb4)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:56.828971 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f433d-5535-42b3-9101-4976b12586ea admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:56.849452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a5ff137b-f65d-4eb4-b09b-d33a51a5aff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6577082bc2e6446591191a439b27de38'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:56.869437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:56.875150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a5ff137b-f65d-4eb4-b09b-d33a51a5aff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6577082bc2e6446591191a439b27de38'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:56.875911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a5ff137b-f65d-4eb4-b09b-d33a51a5aff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6577082bc2e6446591191a439b27de38'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:56.884282 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a5ff137b-f65d-4eb4-b09b-d33a51a5aff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6577082bc2e6446591191a439b27de38'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:56.884943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a5ff137b-f65d-4eb4-b09b-d33a51a5aff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6577082bc2e6446591191a439b27de38'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:56.982689 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1029/2066] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:56 2022] DELETE /identity/v3/users/6577082bc2e6446591191a439b27de38 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:56.988256 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f39c1d-5f75-4944-96ff-faaaa12cbce9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:56.988825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f39c1d-5f75-4944-96ff-faaaa12cbce9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:56.989313 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f39c1d-5f75-4944-96ff-faaaa12cbce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:57.021353 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-541f433d-5535-42b3-9101-4976b12586ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfaf9576d8af4a34aef4e6ecd54d8eb4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:57.037572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-32f39c1d-5f75-4944-96ff-faaaa12cbce9 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:57.042625 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-541f433d-5535-42b3-9101-4976b12586ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfaf9576d8af4a34aef4e6ecd54d8eb4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:57.044600 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-32f39c1d-5f75-4944-96ff-faaaa12cbce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:57.091880 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-541f433d-5535-42b3-9101-4976b12586ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfaf9576d8af4a34aef4e6ecd54d8eb4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:57.093741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-541f433d-5535-42b3-9101-4976b12586ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfaf9576d8af4a34aef4e6ecd54d8eb4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:57.107388 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-541f433d-5535-42b3-9101-4976b12586ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfaf9576d8af4a34aef4e6ecd54d8eb4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:57.131436 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1038/2067] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:56 2022] DELETE /identity/v3/users/bfaf9576d8af4a34aef4e6ecd54d8eb4 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:57.136572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcc530-e435-422a-affe-0198e3d8eb70 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:57.137063 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcc530-e435-422a-affe-0198e3d8eb70 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:57.137365 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcc530-e435-422a-affe-0198e3d8eb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:57.172767 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1030/2068] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:19:57.172963 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3cbcc530-e435-422a-affe-0198e3d8eb70 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:19:57.174349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3cbcc530-e435-422a-affe-0198e3d8eb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:57.179257 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f39c1d-5f75-4944-96ff-faaaa12cbce9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:57.180850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-32f39c1d-5f75-4944-96ff-faaaa12cbce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:57.248713 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1039/2069] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:19:57.251048 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f39c1d-5f75-4944-96ff-faaaa12cbce9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:57.263222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbcc530-e435-422a-affe-0198e3d8eb70 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:57.264880 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3cbcc530-e435-422a-affe-0198e3d8eb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:57.311393 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:57.311665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2b1d6-43da-45a8-86e2-71aa60e10948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:57.312672 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2b1d6-43da-45a8-86e2-71aa60e10948 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:57.313220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2b1d6-43da-45a8-86e2-71aa60e10948 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:57.313682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2b1d6-43da-45a8-86e2-71aa60e10948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:57.333073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2b1d6-43da-45a8-86e2-71aa60e10948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:57.335421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2b1d6-43da-45a8-86e2-71aa60e10948 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:57.349352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1031/2070] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:19:57.354917 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbcc530-e435-422a-affe-0198e3d8eb70 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:57.360541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedec915-267c-4e19-addf-d2c9d16ed4a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:57.361890 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedec915-267c-4e19-addf-d2c9d16ed4a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:57.361890 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedec915-267c-4e19-addf-d2c9d16ed4a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:57.362145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedec915-267c-4e19-addf-d2c9d16ed4a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:57.373234 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedec915-267c-4e19-addf-d2c9d16ed4a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:57.380350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedec915-267c-4e19-addf-d2c9d16ed4a3 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:57.387617 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1040/2071] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:19:58.219239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2b1d6-43da-45a8-86e2-71aa60e10948 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:58.226953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-33b2b1d6-43da-45a8-86e2-71aa60e10948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:58.253830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2b1d6-43da-45a8-86e2-71aa60e10948 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:58.270634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:58.270634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:58.270634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:58.270634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e admin admin] PATH_INFO: `/v3/projects/7b09ad2ebed44a51b1613c02d8f5be03` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:58.270998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b09ad2ebed44a51b1613c02d8f5be03)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:58.272215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:58.280608 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedec915-267c-4e19-addf-d2c9d16ed4a3 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:58.282516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-eedec915-267c-4e19-addf-d2c9d16ed4a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:58.307699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedec915-267c-4e19-addf-d2c9d16ed4a3 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:58.313165 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a97df-e1f9-4aa8-ab94-ea748334a25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:58.315037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a97df-e1f9-4aa8-ab94-ea748334a25c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:58.315037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a97df-e1f9-4aa8-ab94-ea748334a25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:58.315037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a97df-e1f9-4aa8-ab94-ea748334a25c admin admin] PATH_INFO: `/v3/projects/47f832eeb1fd4414b11e5ebb190b3afa` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:58.317907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a97df-e1f9-4aa8-ab94-ea748334a25c admin admin] RBAC: Authorizing `identity:delete_project(project_id=47f832eeb1fd4414b11e5ebb190b3afa)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:58.319202 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a97df-e1f9-4aa8-ab94-ea748334a25c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:58.416688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:58.436081 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b09ad2ebed44a51b1613c02d8f5be03', 'request_id': 'req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.445477 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b09ad2ebed44a51b1613c02d8f5be03', 'request_id': 'req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.470903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f17a97df-e1f9-4aa8-ab94-ea748334a25c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47f832eeb1fd4414b11e5ebb190b3afa', 'request_id': 'req-f17a97df-e1f9-4aa8-ab94-ea748334a25c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.477279 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f17a97df-e1f9-4aa8-ab94-ea748334a25c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47f832eeb1fd4414b11e5ebb190b3afa', 'request_id': 'req-f17a97df-e1f9-4aa8-ab94-ea748334a25c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.501774 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b09ad2ebed44a51b1613c02d8f5be03', 'request_id': 'req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.517617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e admin admin] The token cache is being invalidate because project 7b09ad2ebed44a51b1613c02d8f5be03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:58.518603 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.519476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.521615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1032/2072] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:58 2022] DELETE /identity/v3/projects/7b09ad2ebed44a51b1613c02d8f5be03 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:58.525447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:58.527149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:58.575691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f17a97df-e1f9-4aa8-ab94-ea748334a25c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47f832eeb1fd4414b11e5ebb190b3afa', 'request_id': 'req-f17a97df-e1f9-4aa8-ab94-ea748334a25c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.588950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f17a97df-e1f9-4aa8-ab94-ea748334a25c admin admin] The token cache is being invalidate because project 47f832eeb1fd4414b11e5ebb190b3afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:19:58.589240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f17a97df-e1f9-4aa8-ab94-ea748334a25c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.589952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f17a97df-e1f9-4aa8-ab94-ea748334a25c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.594702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1041/2073] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:58 2022] DELETE /identity/v3/projects/47f832eeb1fd4414b11e5ebb190b3afa => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:58.597054 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a97df-e1f9-4aa8-ab94-ea748334a25c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:58.598826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f17a97df-e1f9-4aa8-ab94-ea748334a25c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:58.609238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f019d-2fe8-4ba0-a6c5-e54b2e40a74e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:58.695823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a97df-e1f9-4aa8-ab94-ea748334a25c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:58.700682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71975fda-dece-491f-97f2-3756a3f3cd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:58.701772 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71975fda-dece-491f-97f2-3756a3f3cd9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:58.702394 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71975fda-dece-491f-97f2-3756a3f3cd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:58.702394 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71975fda-dece-491f-97f2-3756a3f3cd9b admin admin] PATH_INFO: `/v3/users/ac0b94a3bfc648e7ba728b98c3a5dbde` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:58.703554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71975fda-dece-491f-97f2-3756a3f3cd9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac0b94a3bfc648e7ba728b98c3a5dbde)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:58.705954 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9b1c4-13b0-4210-ad6c-9d15d98fc3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:58.706459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71975fda-dece-491f-97f2-3756a3f3cd9b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:58.707210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9b1c4-13b0-4210-ad6c-9d15d98fc3de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:58.707715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9b1c4-13b0-4210-ad6c-9d15d98fc3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:58.708099 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9b1c4-13b0-4210-ad6c-9d15d98fc3de admin admin] PATH_INFO: `/v3/users/4136101ab5f74b96987ff862b070e0ae` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:58.709486 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be9b1c4-13b0-4210-ad6c-9d15d98fc3de admin admin] RBAC: Authorizing `identity:delete_user(user_id=4136101ab5f74b96987ff862b070e0ae)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:58.711704 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be9b1c4-13b0-4210-ad6c-9d15d98fc3de admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:58.936469 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4be9b1c4-13b0-4210-ad6c-9d15d98fc3de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4136101ab5f74b96987ff862b070e0ae'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.936970 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-71975fda-dece-491f-97f2-3756a3f3cd9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac0b94a3bfc648e7ba728b98c3a5dbde'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.944479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:58.953587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4be9b1c4-13b0-4210-ad6c-9d15d98fc3de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4136101ab5f74b96987ff862b070e0ae'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.954656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4be9b1c4-13b0-4210-ad6c-9d15d98fc3de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4136101ab5f74b96987ff862b070e0ae'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.957804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:58.960339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-71975fda-dece-491f-97f2-3756a3f3cd9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac0b94a3bfc648e7ba728b98c3a5dbde'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.970676 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4be9b1c4-13b0-4210-ad6c-9d15d98fc3de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4136101ab5f74b96987ff862b070e0ae'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:58.971737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4be9b1c4-13b0-4210-ad6c-9d15d98fc3de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4136101ab5f74b96987ff862b070e0ae'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:59.060830 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-71975fda-dece-491f-97f2-3756a3f3cd9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac0b94a3bfc648e7ba728b98c3a5dbde'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:59.061634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-71975fda-dece-491f-97f2-3756a3f3cd9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac0b94a3bfc648e7ba728b98c3a5dbde'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:59.069336 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-71975fda-dece-491f-97f2-3756a3f3cd9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac0b94a3bfc648e7ba728b98c3a5dbde'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:19:59.075244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1042/2074] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:58 2022] DELETE /identity/v3/users/ac0b94a3bfc648e7ba728b98c3a5dbde => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:19:59.125345 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1033/2075] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:19:58 2022] DELETE /identity/v3/users/4136101ab5f74b96987ff862b070e0ae => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:19:59.669686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71975fda-dece-491f-97f2-3756a3f3cd9b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:59.671322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-71975fda-dece-491f-97f2-3756a3f3cd9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:59.761195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71975fda-dece-491f-97f2-3756a3f3cd9b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:59.831688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915cd790-4159-45ac-b98e-54a4693616e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:59.832577 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cd790-4159-45ac-b98e-54a4693616e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:59.832859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cd790-4159-45ac-b98e-54a4693616e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:59.833160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cd790-4159-45ac-b98e-54a4693616e7 admin admin] PATH_INFO: `/v3/projects/ef142d225c6540829c8774eba2119508` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:59.834506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915cd790-4159-45ac-b98e-54a4693616e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef142d225c6540829c8774eba2119508)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:59.836484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915cd790-4159-45ac-b98e-54a4693616e7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:59.954229 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9b1c4-13b0-4210-ad6c-9d15d98fc3de admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:59.957229 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4be9b1c4-13b0-4210-ad6c-9d15d98fc3de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:59.988030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9b1c4-13b0-4210-ad6c-9d15d98fc3de admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:00.020100 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-915cd790-4159-45ac-b98e-54a4693616e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef142d225c6540829c8774eba2119508', 'request_id': 'req-915cd790-4159-45ac-b98e-54a4693616e7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:00.057952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-915cd790-4159-45ac-b98e-54a4693616e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef142d225c6540829c8774eba2119508', 'request_id': 'req-915cd790-4159-45ac-b98e-54a4693616e7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:00.190326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a1711-8090-4051-958c-f73ca12a03a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:00.190326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a1711-8090-4051-958c-f73ca12a03a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:00.190326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a1711-8090-4051-958c-f73ca12a03a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:00.190326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a1711-8090-4051-958c-f73ca12a03a4 admin admin] PATH_INFO: `/v3/projects/5ac9864959e3406591785f4f25d66bf9` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:00.191336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a1711-8090-4051-958c-f73ca12a03a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac9864959e3406591785f4f25d66bf9)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:00.192903 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a1711-8090-4051-958c-f73ca12a03a4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:00.210551 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-915cd790-4159-45ac-b98e-54a4693616e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef142d225c6540829c8774eba2119508', 'request_id': 'req-915cd790-4159-45ac-b98e-54a4693616e7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:00.238334 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-915cd790-4159-45ac-b98e-54a4693616e7 admin admin] The token cache is being invalidate because project ef142d225c6540829c8774eba2119508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:00.238559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-915cd790-4159-45ac-b98e-54a4693616e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:00.241665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-915cd790-4159-45ac-b98e-54a4693616e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:00.249330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1043/2076] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:59 2022] DELETE /identity/v3/projects/ef142d225c6540829c8774eba2119508 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:00.250824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79c1ed-87cf-43d8-8407-83255657b55b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:00.251232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79c1ed-87cf-43d8-8407-83255657b55b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:00.251779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79c1ed-87cf-43d8-8407-83255657b55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:00.258446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:00.273369 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bd79c1ed-87cf-43d8-8407-83255657b55b None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:00.274762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bd79c1ed-87cf-43d8-8407-83255657b55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:00.292810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1044/2077] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:00.294402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd5c0e-f428-44af-a560-c606276cbeb6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:00.294749 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd5c0e-f428-44af-a560-c606276cbeb6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:00.295581 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd5c0e-f428-44af-a560-c606276cbeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:00.318050 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-29cd5c0e-f428-44af-a560-c606276cbeb6 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:00.319367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-29cd5c0e-f428-44af-a560-c606276cbeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:00.332335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1045/2078] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:00.333870 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd5c0e-f428-44af-a560-c606276cbeb6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:00.344323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd5c0e-f428-44af-a560-c606276cbeb6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:00.361372 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-137a1711-8090-4051-958c-f73ca12a03a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac9864959e3406591785f4f25d66bf9', 'request_id': 'req-137a1711-8090-4051-958c-f73ca12a03a4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:00.368134 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-137a1711-8090-4051-958c-f73ca12a03a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac9864959e3406591785f4f25d66bf9', 'request_id': 'req-137a1711-8090-4051-958c-f73ca12a03a4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:00.411041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbd2ac-c9f3-4332-a4f7-d90d112be4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:00.411950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbd2ac-c9f3-4332-a4f7-d90d112be4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:00.415328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbd2ac-c9f3-4332-a4f7-d90d112be4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:00.415328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbd2ac-c9f3-4332-a4f7-d90d112be4f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:00.415328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbd2ac-c9f3-4332-a4f7-d90d112be4f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:00.415328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbd2ac-c9f3-4332-a4f7-d90d112be4f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:00.415328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbd2ac-c9f3-4332-a4f7-d90d112be4f6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:00.431322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1046/2079] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:20:00.433127 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-137a1711-8090-4051-958c-f73ca12a03a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac9864959e3406591785f4f25d66bf9', 'request_id': 'req-137a1711-8090-4051-958c-f73ca12a03a4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:00.444706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbd2ac-c9f3-4332-a4f7-d90d112be4f6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:00.451390 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-137a1711-8090-4051-958c-f73ca12a03a4 admin admin] The token cache is being invalidate because project 5ac9864959e3406591785f4f25d66bf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:00.451573 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-137a1711-8090-4051-958c-f73ca12a03a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:00.452838 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-137a1711-8090-4051-958c-f73ca12a03a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:00.456555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1034/2080] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:19:59 2022] DELETE /identity/v3/projects/5ac9864959e3406591785f4f25d66bf9 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:00.462749 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a1711-8090-4051-958c-f73ca12a03a4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:00.465554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbd2ac-c9f3-4332-a4f7-d90d112be4f6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:00.466917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-137a1711-8090-4051-958c-f73ca12a03a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:00.471602 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903d9dc-767f-4285-b163-00b9024b3be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:00.472385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903d9dc-767f-4285-b163-00b9024b3be2 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:00.472869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903d9dc-767f-4285-b163-00b9024b3be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:00.473250 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903d9dc-767f-4285-b163-00b9024b3be2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:00.474280 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903d9dc-767f-4285-b163-00b9024b3be2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:00.476988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903d9dc-767f-4285-b163-00b9024b3be2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:00.495130 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a1711-8090-4051-958c-f73ca12a03a4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:00.502764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad24a-03cb-4f70-8c4b-a1414581cd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:00.503721 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad24a-03cb-4f70-8c4b-a1414581cd79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:00.504007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad24a-03cb-4f70-8c4b-a1414581cd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:00.504285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad24a-03cb-4f70-8c4b-a1414581cd79 admin admin] PATH_INFO: `/v3/users/ea6ac4c675ad4245b25f1a1b873712de` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:00.505554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ad24a-03cb-4f70-8c4b-a1414581cd79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea6ac4c675ad4245b25f1a1b873712de)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:00.507555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ad24a-03cb-4f70-8c4b-a1414581cd79 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:00.584300 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1047/2081] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:00 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:00.588883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903d9dc-767f-4285-b163-00b9024b3be2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:00.591549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f903d9dc-767f-4285-b163-00b9024b3be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:00.656086 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a84ad24a-03cb-4f70-8c4b-a1414581cd79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea6ac4c675ad4245b25f1a1b873712de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:00.671312 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a84ad24a-03cb-4f70-8c4b-a1414581cd79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea6ac4c675ad4245b25f1a1b873712de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:00.673687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a84ad24a-03cb-4f70-8c4b-a1414581cd79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea6ac4c675ad4245b25f1a1b873712de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:00.679054 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=17 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:00.683048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903d9dc-767f-4285-b163-00b9024b3be2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:00.684220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a84ad24a-03cb-4f70-8c4b-a1414581cd79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea6ac4c675ad4245b25f1a1b873712de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:00.685403 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a84ad24a-03cb-4f70-8c4b-a1414581cd79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea6ac4c675ad4245b25f1a1b873712de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:00.691166 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfe784-b16d-4cd3-9dec-1f4266b3ff6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:00.691905 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe784-b16d-4cd3-9dec-1f4266b3ff6a admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:00.692286 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe784-b16d-4cd3-9dec-1f4266b3ff6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:00.692676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe784-b16d-4cd3-9dec-1f4266b3ff6a admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:00.693995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfe784-b16d-4cd3-9dec-1f4266b3ff6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:00.696306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfe784-b16d-4cd3-9dec-1f4266b3ff6a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:00.768817 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1035/2082] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:00 2022] DELETE /identity/v3/users/ea6ac4c675ad4245b25f1a1b873712de => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:00.788656 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1048/2083] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:00 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:00.789094 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad24a-03cb-4f70-8c4b-a1414581cd79 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:00.790791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a84ad24a-03cb-4f70-8c4b-a1414581cd79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:00.851408 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfe784-b16d-4cd3-9dec-1f4266b3ff6a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:00.853359 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fccfe784-b16d-4cd3-9dec-1f4266b3ff6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:00.879784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfe784-b16d-4cd3-9dec-1f4266b3ff6a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:00.904784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad24a-03cb-4f70-8c4b-a1414581cd79 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:00.913708 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cc78e-844b-4c64-bbc4-e3bb548094f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:00.914829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cc78e-844b-4c64-bbc4-e3bb548094f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:00.914829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cc78e-844b-4c64-bbc4-e3bb548094f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:00.915093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cc78e-844b-4c64-bbc4-e3bb548094f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:00.915857 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390cc78e-844b-4c64-bbc4-e3bb548094f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:00.917580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390cc78e-844b-4c64-bbc4-e3bb548094f4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:00.929569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1036/2084] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:00.933373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cc78e-844b-4c64-bbc4-e3bb548094f4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:00.944063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cc78e-844b-4c64-bbc4-e3bb548094f4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:00.948573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9099e-8b5d-4fb2-a513-f689ba0c2483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:00.948714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0ee7e-9671-437b-8aa4-0d61a944d258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:00.949369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0ee7e-9671-437b-8aa4-0d61a944d258 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:00.949467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9099e-8b5d-4fb2-a513-f689ba0c2483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:00.949655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0ee7e-9671-437b-8aa4-0d61a944d258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:00.949783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9099e-8b5d-4fb2-a513-f689ba0c2483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:00.949934 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0ee7e-9671-437b-8aa4-0d61a944d258 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:00.950071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9099e-8b5d-4fb2-a513-f689ba0c2483 admin admin] PATH_INFO: `/v3/users/78195ff60f0242ada974a8e08d1f2754` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:00.950661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0ee7e-9671-437b-8aa4-0d61a944d258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:00.951328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9099e-8b5d-4fb2-a513-f689ba0c2483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78195ff60f0242ada974a8e08d1f2754)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:00.952285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0ee7e-9671-437b-8aa4-0d61a944d258 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:00.953457 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9099e-8b5d-4fb2-a513-f689ba0c2483 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:00.962770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1037/2085] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:00.966661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0ee7e-9671-437b-8aa4-0d61a944d258 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:00.977336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0ee7e-9671-437b-8aa4-0d61a944d258 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:00.982057 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f858c6b-6d6f-4048-9ae0-d139751f2fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:00.982935 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f858c6b-6d6f-4048-9ae0-d139751f2fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:00.983262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f858c6b-6d6f-4048-9ae0-d139751f2fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:00.983545 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f858c6b-6d6f-4048-9ae0-d139751f2fe8 admin admin] PATH_INFO: `/v3/projects/4afcf1169dca4aa7b0c2c4160c0bd743/users/52e999e636af44b79e9f64aff5b5294a/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:00.998866 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f858c6b-6d6f-4048-9ae0-d139751f2fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4afcf1169dca4aa7b0c2c4160c0bd743, user_id=52e999e636af44b79e9f64aff5b5294a, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:01.001571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f858c6b-6d6f-4048-9ae0-d139751f2fe8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:01.089314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1038/2086] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:00 2022] PUT /identity/v3/projects/4afcf1169dca4aa7b0c2c4160c0bd743/users/52e999e636af44b79e9f64aff5b5294a/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:01.100988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d52a8f-4b52-4556-ae37-eb6f1ff506d9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:01.101134 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d52a8f-4b52-4556-ae37-eb6f1ff506d9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:01.101580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d52a8f-4b52-4556-ae37-eb6f1ff506d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:01.152444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-88d52a8f-4b52-4556-ae37-eb6f1ff506d9 None None] MFA Rules not processed for user `52e999e636af44b79e9f64aff5b5294a`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:01.154351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-88d52a8f-4b52-4556-ae37-eb6f1ff506d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:01.209850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-59f9099e-8b5d-4fb2-a513-f689ba0c2483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78195ff60f0242ada974a8e08d1f2754'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:01.225151 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-59f9099e-8b5d-4fb2-a513-f689ba0c2483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78195ff60f0242ada974a8e08d1f2754'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:01.240621 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1039/2087] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:01.245153 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d52a8f-4b52-4556-ae37-eb6f1ff506d9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:01.300687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-59f9099e-8b5d-4fb2-a513-f689ba0c2483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78195ff60f0242ada974a8e08d1f2754'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:01.301395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-59f9099e-8b5d-4fb2-a513-f689ba0c2483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78195ff60f0242ada974a8e08d1f2754'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:01.308893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-59f9099e-8b5d-4fb2-a513-f689ba0c2483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78195ff60f0242ada974a8e08d1f2754'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:01.312049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d52a8f-4b52-4556-ae37-eb6f1ff506d9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:01.315131 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1049/2088] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:00 2022] DELETE /identity/v3/users/78195ff60f0242ada974a8e08d1f2754 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:01.319908 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673362a6-fa66-4b5e-adb2-c472cecfa357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:01.320575 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673362a6-fa66-4b5e-adb2-c472cecfa357 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:01.320859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673362a6-fa66-4b5e-adb2-c472cecfa357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:01.321177 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673362a6-fa66-4b5e-adb2-c472cecfa357 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:01.321859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673362a6-fa66-4b5e-adb2-c472cecfa357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:01.323803 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673362a6-fa66-4b5e-adb2-c472cecfa357 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:01.355101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9099e-8b5d-4fb2-a513-f689ba0c2483 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:01.356820 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-59f9099e-8b5d-4fb2-a513-f689ba0c2483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:01.434847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1040/2089] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:01 2022] POST /identity/v3/projects => generated 383 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:01.439730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673362a6-fa66-4b5e-adb2-c472cecfa357 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:01.441772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-673362a6-fa66-4b5e-adb2-c472cecfa357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:01.456170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9099e-8b5d-4fb2-a513-f689ba0c2483 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:01.520952 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673362a6-fa66-4b5e-adb2-c472cecfa357 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:01.526059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870efc2b-2d2b-433a-8b8e-4f4a162c52f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:01.526903 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870efc2b-2d2b-433a-8b8e-4f4a162c52f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:01.527206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870efc2b-2d2b-433a-8b8e-4f4a162c52f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:01.527401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fee2a-e6d3-4b95-af63-acc209b71710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:01.527519 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870efc2b-2d2b-433a-8b8e-4f4a162c52f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:01.528202 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870efc2b-2d2b-433a-8b8e-4f4a162c52f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:01.528315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fee2a-e6d3-4b95-af63-acc209b71710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:01.528540 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fee2a-e6d3-4b95-af63-acc209b71710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:01.528818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fee2a-e6d3-4b95-af63-acc209b71710 admin admin] PATH_INFO: `/v3/projects/ed6e5b938daa4e478ef9d1ed093efbe7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:01.530260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fee2a-e6d3-4b95-af63-acc209b71710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed6e5b938daa4e478ef9d1ed093efbe7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:01.530434 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870efc2b-2d2b-433a-8b8e-4f4a162c52f5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:01.532198 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fee2a-e6d3-4b95-af63-acc209b71710 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:01.614027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1041/2090] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:01 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:01.617743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870efc2b-2d2b-433a-8b8e-4f4a162c52f5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:01.627503 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870efc2b-2d2b-433a-8b8e-4f4a162c52f5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:01.632409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13898345-ee3d-41e4-b9be-383910fde21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:01.633045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13898345-ee3d-41e4-b9be-383910fde21d admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:01.633327 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13898345-ee3d-41e4-b9be-383910fde21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:01.633602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13898345-ee3d-41e4-b9be-383910fde21d admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:01.634314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13898345-ee3d-41e4-b9be-383910fde21d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:01.635953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13898345-ee3d-41e4-b9be-383910fde21d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:01.646299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1042/2091] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:01.650165 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13898345-ee3d-41e4-b9be-383910fde21d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:01.660741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13898345-ee3d-41e4-b9be-383910fde21d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:01.666696 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-551fee2a-e6d3-4b95-af63-acc209b71710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed6e5b938daa4e478ef9d1ed093efbe7', 'request_id': 'req-551fee2a-e6d3-4b95-af63-acc209b71710'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:01.679155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-551fee2a-e6d3-4b95-af63-acc209b71710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed6e5b938daa4e478ef9d1ed093efbe7', 'request_id': 'req-551fee2a-e6d3-4b95-af63-acc209b71710'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:01.722234 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbe1af-be8e-4289-8365-f9a3c557f1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:01.722904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbe1af-be8e-4289-8365-f9a3c557f1cc admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:01.723191 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbe1af-be8e-4289-8365-f9a3c557f1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:01.723494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbe1af-be8e-4289-8365-f9a3c557f1cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:01.724213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbe1af-be8e-4289-8365-f9a3c557f1cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:01.725817 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbe1af-be8e-4289-8365-f9a3c557f1cc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:01.735782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1043/2092] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:01.736585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbe1af-be8e-4289-8365-f9a3c557f1cc admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:01.738291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c6bbe1af-be8e-4289-8365-f9a3c557f1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:01.750450 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbe1af-be8e-4289-8365-f9a3c557f1cc admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:01.755756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d449e-6f0d-4956-99b9-5e0b08f32fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:01.756592 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d449e-6f0d-4956-99b9-5e0b08f32fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:01.756873 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d449e-6f0d-4956-99b9-5e0b08f32fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:01.757145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d449e-6f0d-4956-99b9-5e0b08f32fda admin admin] PATH_INFO: `/v3/projects/d6e6bc2109c44c37a0ed635b021bba7e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:01.758329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d449e-6f0d-4956-99b9-5e0b08f32fda admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6e6bc2109c44c37a0ed635b021bba7e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:01.760228 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d449e-6f0d-4956-99b9-5e0b08f32fda admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:01.798261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-551fee2a-e6d3-4b95-af63-acc209b71710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed6e5b938daa4e478ef9d1ed093efbe7', 'request_id': 'req-551fee2a-e6d3-4b95-af63-acc209b71710'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:01.830588 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-551fee2a-e6d3-4b95-af63-acc209b71710 admin admin] The token cache is being invalidate because project ed6e5b938daa4e478ef9d1ed093efbe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:01.830588 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-551fee2a-e6d3-4b95-af63-acc209b71710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:01.830588 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-551fee2a-e6d3-4b95-af63-acc209b71710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:01.830588 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1050/2093] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:01 2022] DELETE /identity/v3/projects/ed6e5b938daa4e478ef9d1ed093efbe7 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:01.830588 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fee2a-e6d3-4b95-af63-acc209b71710 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:01.836064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-551fee2a-e6d3-4b95-af63-acc209b71710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:01.851388 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fee2a-e6d3-4b95-af63-acc209b71710 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:01.858128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8d140-4dd7-4539-bfd4-b2c9a69cb0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:01.858128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8d140-4dd7-4539-bfd4-b2c9a69cb0db admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:01.858128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8d140-4dd7-4539-bfd4-b2c9a69cb0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:01.858128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8d140-4dd7-4539-bfd4-b2c9a69cb0db admin admin] PATH_INFO: `/v3/projects/db8d818144084415bb59bdcc6cb53712/users/3c4a91082e7549938ddd755e1a1a75c4/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:01.875321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8d140-4dd7-4539-bfd4-b2c9a69cb0db admin admin] RBAC: Authorizing `identity:create_grant(project_id=db8d818144084415bb59bdcc6cb53712, user_id=3c4a91082e7549938ddd755e1a1a75c4, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:01.879019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8d140-4dd7-4539-bfd4-b2c9a69cb0db admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:01.924407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-831d449e-6f0d-4956-99b9-5e0b08f32fda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6e6bc2109c44c37a0ed635b021bba7e', 'request_id': 'req-831d449e-6f0d-4956-99b9-5e0b08f32fda'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:01.934763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-831d449e-6f0d-4956-99b9-5e0b08f32fda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6e6bc2109c44c37a0ed635b021bba7e', 'request_id': 'req-831d449e-6f0d-4956-99b9-5e0b08f32fda'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:01.990142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1051/2094] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:01 2022] PUT /identity/v3/projects/db8d818144084415bb59bdcc6cb53712/users/3c4a91082e7549938ddd755e1a1a75c4/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:01.990142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88897684-30f3-45bb-91d3-303a0f814546 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:01.990455 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88897684-30f3-45bb-91d3-303a0f814546 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:01.990695 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88897684-30f3-45bb-91d3-303a0f814546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.009969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-831d449e-6f0d-4956-99b9-5e0b08f32fda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6e6bc2109c44c37a0ed635b021bba7e', 'request_id': 'req-831d449e-6f0d-4956-99b9-5e0b08f32fda'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:02.010751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-88897684-30f3-45bb-91d3-303a0f814546 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:02.012117 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-88897684-30f3-45bb-91d3-303a0f814546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:02.022723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-831d449e-6f0d-4956-99b9-5e0b08f32fda admin admin] The token cache is being invalidate because project d6e6bc2109c44c37a0ed635b021bba7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:02.025059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-831d449e-6f0d-4956-99b9-5e0b08f32fda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:02.025059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-831d449e-6f0d-4956-99b9-5e0b08f32fda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:02.027222 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1044/2095] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:01 2022] DELETE /identity/v3/projects/d6e6bc2109c44c37a0ed635b021bba7e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:02.027432 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b64d58-c6fb-42a1-81cc-f810be1917ea None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.028045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b64d58-c6fb-42a1-81cc-f810be1917ea None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.028635 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b64d58-c6fb-42a1-81cc-f810be1917ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.072351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-85b64d58-c6fb-42a1-81cc-f810be1917ea None None] MFA Rules not processed for user `3c4a91082e7549938ddd755e1a1a75c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:02.073626 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-85b64d58-c6fb-42a1-81cc-f810be1917ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:02.092919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1052/2096] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:02.092919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b67606-3087-4d57-a663-c41cd90e65f4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.092919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b67606-3087-4d57-a663-c41cd90e65f4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.092919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b67606-3087-4d57-a663-c41cd90e65f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.114135 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a0b67606-3087-4d57-a663-c41cd90e65f4 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:02.115394 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a0b67606-3087-4d57-a663-c41cd90e65f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:02.123724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1053/2097] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:02.124956 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ecc31-e597-4979-8738-c7d9e7617d68 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.125241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ecc31-e597-4979-8738-c7d9e7617d68 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.125514 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ecc31-e597-4979-8738-c7d9e7617d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.143760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0a7ecc31-e597-4979-8738-c7d9e7617d68 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:02.144938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0a7ecc31-e597-4979-8738-c7d9e7617d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:02.153123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1054/2098] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:02.154570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc057ecd-2a59-4090-b07e-2cb34be54899 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.154716 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc057ecd-2a59-4090-b07e-2cb34be54899 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.155134 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc057ecd-2a59-4090-b07e-2cb34be54899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.158331 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1045/2099] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:02.159162 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b64d58-c6fb-42a1-81cc-f810be1917ea None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.170079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b64d58-c6fb-42a1-81cc-f810be1917ea None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.175233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-dc057ecd-2a59-4090-b07e-2cb34be54899 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:02.175496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b087a-e1a1-4c87-a806-2a33a9dcfcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.176156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b087a-e1a1-4c87-a806-2a33a9dcfcb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.176445 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b087a-e1a1-4c87-a806-2a33a9dcfcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.176555 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dc057ecd-2a59-4090-b07e-2cb34be54899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:02.176730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b087a-e1a1-4c87-a806-2a33a9dcfcb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.177457 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b087a-e1a1-4c87-a806-2a33a9dcfcb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:02.177768 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b087a-e1a1-4c87-a806-2a33a9dcfcb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.179386 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b087a-e1a1-4c87-a806-2a33a9dcfcb8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.184389 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1055/2100] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:02.185170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc057ecd-2a59-4090-b07e-2cb34be54899 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.186432 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dc057ecd-2a59-4090-b07e-2cb34be54899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:02.195341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1046/2101] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:20:02.195341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b087a-e1a1-4c87-a806-2a33a9dcfcb8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.199219 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc057ecd-2a59-4090-b07e-2cb34be54899 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.203369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b087a-e1a1-4c87-a806-2a33a9dcfcb8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.203774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795beca-841a-41e8-8dbb-e74aaf118707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.204861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795beca-841a-41e8-8dbb-e74aaf118707 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.205142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795beca-841a-41e8-8dbb-e74aaf118707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.205419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795beca-841a-41e8-8dbb-e74aaf118707 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.206103 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795beca-841a-41e8-8dbb-e74aaf118707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.208055 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795beca-841a-41e8-8dbb-e74aaf118707 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.208237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544e780-b2c9-4746-905e-0c1a42e5ca8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.208882 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544e780-b2c9-4746-905e-0c1a42e5ca8b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.209164 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544e780-b2c9-4746-905e-0c1a42e5ca8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.209441 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544e780-b2c9-4746-905e-0c1a42e5ca8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.210157 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544e780-b2c9-4746-905e-0c1a42e5ca8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:02.210464 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544e780-b2c9-4746-905e-0c1a42e5ca8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.212061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544e780-b2c9-4746-905e-0c1a42e5ca8b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.224752 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1047/2102] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:20:02.225577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544e780-b2c9-4746-905e-0c1a42e5ca8b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.236988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544e780-b2c9-4746-905e-0c1a42e5ca8b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.241608 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80a14d-7885-401e-b842-ac001b1f9a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.242266 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80a14d-7885-401e-b842-ac001b1f9a34 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.242553 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80a14d-7885-401e-b842-ac001b1f9a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.242837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80a14d-7885-401e-b842-ac001b1f9a34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.243597 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80a14d-7885-401e-b842-ac001b1f9a34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.245551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80a14d-7885-401e-b842-ac001b1f9a34 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.291242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1056/2103] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:02 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:02.292075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795beca-841a-41e8-8dbb-e74aaf118707 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.352304 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795beca-841a-41e8-8dbb-e74aaf118707 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.352998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1048/2104] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:02 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:02.357037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80a14d-7885-401e-b842-ac001b1f9a34 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.408534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03add886-4707-438a-941c-d6b29b12573f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.409307 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03add886-4707-438a-941c-d6b29b12573f admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.409594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03add886-4707-438a-941c-d6b29b12573f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.409871 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03add886-4707-438a-941c-d6b29b12573f admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.410596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03add886-4707-438a-941c-d6b29b12573f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.412800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03add886-4707-438a-941c-d6b29b12573f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.419548 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80a14d-7885-401e-b842-ac001b1f9a34 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.425329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e282693-f345-4285-8bcb-2247d20e277f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.426023 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e282693-f345-4285-8bcb-2247d20e277f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.426309 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e282693-f345-4285-8bcb-2247d20e277f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.426593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e282693-f345-4285-8bcb-2247d20e277f admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.427313 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e282693-f345-4285-8bcb-2247d20e277f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.429226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e282693-f345-4285-8bcb-2247d20e277f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.500608 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1057/2105] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:02 2022] POST /identity/v3/projects => generated 371 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:02.501472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03add886-4707-438a-941c-d6b29b12573f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.519818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:02.527974 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1049/2106] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:02 2022] POST /identity/v3/users => generated 299 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:02.527974 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e282693-f345-4285-8bcb-2247d20e277f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.537075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e282693-f345-4285-8bcb-2247d20e277f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.563777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03add886-4707-438a-941c-d6b29b12573f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.569823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315ec07-c69f-46e1-93d4-bd6d8ef8c61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.570548 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315ec07-c69f-46e1-93d4-bd6d8ef8c61f admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.570831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315ec07-c69f-46e1-93d4-bd6d8ef8c61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.571217 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315ec07-c69f-46e1-93d4-bd6d8ef8c61f admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.572056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315ec07-c69f-46e1-93d4-bd6d8ef8c61f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.573792 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315ec07-c69f-46e1-93d4-bd6d8ef8c61f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.596929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b04db-e996-4be0-8bdd-ed51c7ffe675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.597641 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b04db-e996-4be0-8bdd-ed51c7ffe675 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.597928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b04db-e996-4be0-8bdd-ed51c7ffe675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.598211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b04db-e996-4be0-8bdd-ed51c7ffe675 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.598929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b04db-e996-4be0-8bdd-ed51c7ffe675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.601005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b04db-e996-4be0-8bdd-ed51c7ffe675 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.642874 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1058/2107] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:02 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:02.642874 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315ec07-c69f-46e1-93d4-bd6d8ef8c61f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.655523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315ec07-c69f-46e1-93d4-bd6d8ef8c61f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.661522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae0b34-11a8-4d21-a6df-f737e305efd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.662350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae0b34-11a8-4d21-a6df-f737e305efd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.662705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae0b34-11a8-4d21-a6df-f737e305efd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.663055 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae0b34-11a8-4d21-a6df-f737e305efd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.663906 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae0b34-11a8-4d21-a6df-f737e305efd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.665746 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae0b34-11a8-4d21-a6df-f737e305efd1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.680221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1050/2108] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:02 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:02.681720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b04db-e996-4be0-8bdd-ed51c7ffe675 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.693652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b04db-e996-4be0-8bdd-ed51c7ffe675 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.698321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5be3a-05d7-4395-a035-431e3d28929e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.702390 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5be3a-05d7-4395-a035-431e3d28929e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.702528 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1059/2109] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:02.702647 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5be3a-05d7-4395-a035-431e3d28929e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.702647 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5be3a-05d7-4395-a035-431e3d28929e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.702806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae0b34-11a8-4d21-a6df-f737e305efd1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.702890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5be3a-05d7-4395-a035-431e3d28929e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.702890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5be3a-05d7-4395-a035-431e3d28929e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.712769 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1051/2110] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:02.713647 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5be3a-05d7-4395-a035-431e3d28929e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.718492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae0b34-11a8-4d21-a6df-f737e305efd1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.724463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8296e-81aa-48bf-a4ae-f31d6ae5e2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.724600 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5be3a-05d7-4395-a035-431e3d28929e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.725273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8296e-81aa-48bf-a4ae-f31d6ae5e2cf admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.725808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8296e-81aa-48bf-a4ae-f31d6ae5e2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.726147 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8296e-81aa-48bf-a4ae-f31d6ae5e2cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.726939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8296e-81aa-48bf-a4ae-f31d6ae5e2cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.727554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:02.730678 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8296e-81aa-48bf-a4ae-f31d6ae5e2cf admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.732187 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da043f4-ff33-465a-996a-9f9c229ea904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.733259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da043f4-ff33-465a-996a-9f9c229ea904 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.733731 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da043f4-ff33-465a-996a-9f9c229ea904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.734210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da043f4-ff33-465a-996a-9f9c229ea904 admin admin] PATH_INFO: `/v3/projects/b359beefcd8943a8a7761abc331e6c5a/users/bcb3b268867840e7b8e1390379291f43/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.741769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1060/2111] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:02.742668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8296e-81aa-48bf-a4ae-f31d6ae5e2cf admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.752795 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da043f4-ff33-465a-996a-9f9c229ea904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b359beefcd8943a8a7761abc331e6c5a, user_id=bcb3b268867840e7b8e1390379291f43, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.753072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8296e-81aa-48bf-a4ae-f31d6ae5e2cf admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.755712 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da043f4-ff33-465a-996a-9f9c229ea904 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.763279 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e056e-724a-468e-b970-bd060a27a57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.764059 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e056e-724a-468e-b970-bd060a27a57d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.765117 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e056e-724a-468e-b970-bd060a27a57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.765117 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e056e-724a-468e-b970-bd060a27a57d admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.765491 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e056e-724a-468e-b970-bd060a27a57d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.767324 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e056e-724a-468e-b970-bd060a27a57d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.777637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1061/2112] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:02.778440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e056e-724a-468e-b970-bd060a27a57d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.789671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e056e-724a-468e-b970-bd060a27a57d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.794388 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6845b-2013-4cd7-b3d4-917793763790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.795097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6845b-2013-4cd7-b3d4-917793763790 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.795416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6845b-2013-4cd7-b3d4-917793763790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.795696 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6845b-2013-4cd7-b3d4-917793763790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.796419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6845b-2013-4cd7-b3d4-917793763790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.798060 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6845b-2013-4cd7-b3d4-917793763790 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.808537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1062/2113] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:02.809327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6845b-2013-4cd7-b3d4-917793763790 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.819021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6845b-2013-4cd7-b3d4-917793763790 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.823723 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259b91c-ab18-490d-8cb5-1a4b91343810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.824608 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259b91c-ab18-490d-8cb5-1a4b91343810 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.824908 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259b91c-ab18-490d-8cb5-1a4b91343810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.825189 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259b91c-ab18-490d-8cb5-1a4b91343810 admin admin] PATH_INFO: `/v3/projects/d1d278a02d924d9f990bdf0bac334b05/users/5143a5d2d2e84d8396174ae3dc505c54/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.827845 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1052/2114] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:02 2022] PUT /identity/v3/projects/b359beefcd8943a8a7761abc331e6c5a/users/bcb3b268867840e7b8e1390379291f43/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:02.828941 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da043f4-ff33-465a-996a-9f9c229ea904 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.841276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259b91c-ab18-490d-8cb5-1a4b91343810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d278a02d924d9f990bdf0bac334b05, user_id=5143a5d2d2e84d8396174ae3dc505c54, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.843948 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259b91c-ab18-490d-8cb5-1a4b91343810 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.893163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da043f4-ff33-465a-996a-9f9c229ea904 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.898013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5997b-134e-47d8-a271-55844f9ec7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:02.898901 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5997b-134e-47d8-a271-55844f9ec7d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:02.901262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5997b-134e-47d8-a271-55844f9ec7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:02.901262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5997b-134e-47d8-a271-55844f9ec7d4 admin admin] PATH_INFO: `/v3/projects/301009c4e160494d97d1913a2ae3a0d7/users/b3308f84d204454681b15647e0eaf059/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:02.914172 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5997b-134e-47d8-a271-55844f9ec7d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=301009c4e160494d97d1913a2ae3a0d7, user_id=b3308f84d204454681b15647e0eaf059, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:02.916893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5997b-134e-47d8-a271-55844f9ec7d4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:02.924056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1063/2115] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:02 2022] PUT /identity/v3/projects/d1d278a02d924d9f990bdf0bac334b05/users/5143a5d2d2e84d8396174ae3dc505c54/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:02.925478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259b91c-ab18-490d-8cb5-1a4b91343810 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:02.943025 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:02.996916 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259b91c-ab18-490d-8cb5-1a4b91343810 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:02.999076 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1053/2116] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:02 2022] PUT /identity/v3/projects/301009c4e160494d97d1913a2ae3a0d7/users/b3308f84d204454681b15647e0eaf059/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:03.000224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d969-fbed-4374-aefe-df792e20b0f3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:03.000518 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d969-fbed-4374-aefe-df792e20b0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:03.000805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d969-fbed-4374-aefe-df792e20b0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:03.035162 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a7c4d969-fbed-4374-aefe-df792e20b0f3 None None] MFA Rules not processed for user `5143a5d2d2e84d8396174ae3dc505c54`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:03.036574 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a7c4d969-fbed-4374-aefe-df792e20b0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:03.053447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff07608-3d25-4193-86cc-2954c020fb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:03.054115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff07608-3d25-4193-86cc-2954c020fb48 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:03.054394 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff07608-3d25-4193-86cc-2954c020fb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:03.054666 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff07608-3d25-4193-86cc-2954c020fb48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:03.055426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff07608-3d25-4193-86cc-2954c020fb48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:03.057008 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff07608-3d25-4193-86cc-2954c020fb48 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:03.067952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1064/2117] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:03.069289 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46108e4d-71af-4028-872a-375cd11485ad None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:03.069650 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46108e4d-71af-4028-872a-375cd11485ad None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:03.070002 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46108e4d-71af-4028-872a-375cd11485ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:03.101388 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-46108e4d-71af-4028-872a-375cd11485ad None None] MFA Rules not processed for user `b3308f84d204454681b15647e0eaf059`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:03.102642 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-46108e4d-71af-4028-872a-375cd11485ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:03.106936 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1054/2118] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:03.107815 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4d969-fbed-4374-aefe-df792e20b0f3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:03.119384 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4d969-fbed-4374-aefe-df792e20b0f3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:03.128016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c865b4-6144-4201-b2d2-1b6fa0577ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:03.129410 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c865b4-6144-4201-b2d2-1b6fa0577ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:03.129752 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c865b4-6144-4201-b2d2-1b6fa0577ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:03.130664 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c865b4-6144-4201-b2d2-1b6fa0577ec4 admin admin] PATH_INFO: `/v3/projects/b359beefcd8943a8a7761abc331e6c5a/users/bcb3b268867840e7b8e1390379291f43/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:03.135248 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c865b4-6144-4201-b2d2-1b6fa0577ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b359beefcd8943a8a7761abc331e6c5a, user_id=bcb3b268867840e7b8e1390379291f43, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:03.140292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c865b4-6144-4201-b2d2-1b6fa0577ec4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:03.147355 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:03.163919 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1055/2119] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:03 2022] PUT /identity/v3/projects/b359beefcd8943a8a7761abc331e6c5a/users/bcb3b268867840e7b8e1390379291f43/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:03.168640 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c865b4-6144-4201-b2d2-1b6fa0577ec4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:03.197622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1065/2120] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:03.199321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8e8af-f7c9-4d24-9c60-009247d75541 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:03.199677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8e8af-f7c9-4d24-9c60-009247d75541 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:03.200340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8e8af-f7c9-4d24-9c60-009247d75541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:03.236427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f1b8e8af-f7c9-4d24-9c60-009247d75541 None None] MFA Rules not processed for user `bcb3b268867840e7b8e1390379291f43`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:03.238225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f1b8e8af-f7c9-4d24-9c60-009247d75541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:03.263684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c865b4-6144-4201-b2d2-1b6fa0577ec4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:03.269898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ae748-ccfd-4f45-8714-8b11f3c1ee3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:03.270693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ae748-ccfd-4f45-8714-8b11f3c1ee3e admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:03.271150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ae748-ccfd-4f45-8714-8b11f3c1ee3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:03.271524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ae748-ccfd-4f45-8714-8b11f3c1ee3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:03.272301 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ae748-ccfd-4f45-8714-8b11f3c1ee3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:03.274581 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ae748-ccfd-4f45-8714-8b11f3c1ee3e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:03.335082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1066/2121] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:03.339348 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8e8af-f7c9-4d24-9c60-009247d75541 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:03.339348 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f1b8e8af-f7c9-4d24-9c60-009247d75541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:03.385155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1056/2122] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:03 2022] POST /identity/v3/projects => generated 383 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:03.385155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ae748-ccfd-4f45-8714-8b11f3c1ee3e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:03.385155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f95ae748-ccfd-4f45-8714-8b11f3c1ee3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:03.415487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8e8af-f7c9-4d24-9c60-009247d75541 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:03.444128 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ae748-ccfd-4f45-8714-8b11f3c1ee3e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:03.450570 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943ac45-60aa-44c8-9e38-dd37c51adb2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:03.451267 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943ac45-60aa-44c8-9e38-dd37c51adb2c service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:03.451555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943ac45-60aa-44c8-9e38-dd37c51adb2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:03.452201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943ac45-60aa-44c8-9e38-dd37c51adb2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:03.454289 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9943ac45-60aa-44c8-9e38-dd37c51adb2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:03.480110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca2265-ad4e-4b86-9930-d03cc704ecf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:03.480798 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca2265-ad4e-4b86-9930-d03cc704ecf2 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:03.481086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca2265-ad4e-4b86-9930-d03cc704ecf2 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:03.481364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca2265-ad4e-4b86-9930-d03cc704ecf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:03.509081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:03.521522 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9943ac45-60aa-44c8-9e38-dd37c51adb2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:03.522974 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9943ac45-60aa-44c8-9e38-dd37c51adb2c service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:03.533137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1057/2123] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:03.533790 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943ac45-60aa-44c8-9e38-dd37c51adb2c service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:03.540141 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca2265-ad4e-4b86-9930-d03cc704ecf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:03.541700 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca2265-ad4e-4b86-9930-d03cc704ecf2 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:03.551806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1067/2124] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:03.602807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943ac45-60aa-44c8-9e38-dd37c51adb2c service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:03.612532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba532f1-ece9-4731-b058-bea7a8944b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:03.613212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba532f1-ece9-4731-b058-bea7a8944b5e admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:03.613485 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba532f1-ece9-4731-b058-bea7a8944b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:03.613760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba532f1-ece9-4731-b058-bea7a8944b5e admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:03.614558 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba532f1-ece9-4731-b058-bea7a8944b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:03.616402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba532f1-ece9-4731-b058-bea7a8944b5e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:03.711092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1058/2125] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:03 2022] POST /identity/v3/users => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:03.712320 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca2265-ad4e-4b86-9930-d03cc704ecf2 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:03.723634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca2265-ad4e-4b86-9930-d03cc704ecf2 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:03.723878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:03.728622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b5da9-67eb-4330-ab80-45594d6c8c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:03.729334 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b5da9-67eb-4330-ab80-45594d6c8c12 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:03.729614 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b5da9-67eb-4330-ab80-45594d6c8c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:03.729891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b5da9-67eb-4330-ab80-45594d6c8c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:03.730620 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b5da9-67eb-4330-ab80-45594d6c8c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:03.732254 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b5da9-67eb-4330-ab80-45594d6c8c12 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:03.742310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1068/2126] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:03.746611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba532f1-ece9-4731-b058-bea7a8944b5e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:03.756780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba532f1-ece9-4731-b058-bea7a8944b5e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:03.761361 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c256de-0842-4e08-b0cc-d5b28502f0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:03.762233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c256de-0842-4e08-b0cc-d5b28502f0c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:03.762516 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c256de-0842-4e08-b0cc-d5b28502f0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:03.762792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c256de-0842-4e08-b0cc-d5b28502f0c5 admin admin] PATH_INFO: `/v3/projects/12978f20d301496881b11bc3ba8bf5e2/users/3cc67573bb2b491b9c82b3657e5eed03/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:03.778423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c256de-0842-4e08-b0cc-d5b28502f0c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12978f20d301496881b11bc3ba8bf5e2, user_id=3cc67573bb2b491b9c82b3657e5eed03, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:03.781574 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c256de-0842-4e08-b0cc-d5b28502f0c5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:03.902473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1059/2127] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:03 2022] PUT /identity/v3/projects/12978f20d301496881b11bc3ba8bf5e2/users/3cc67573bb2b491b9c82b3657e5eed03/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:03.906719 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b5da9-67eb-4330-ab80-45594d6c8c12 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:03.926349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b5da9-67eb-4330-ab80-45594d6c8c12 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:03.992839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bdfec-ac13-42d3-a320-5dc5ee659903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:03.993514 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bdfec-ac13-42d3-a320-5dc5ee659903 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:03.993814 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bdfec-ac13-42d3-a320-5dc5ee659903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:03.994065 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bdfec-ac13-42d3-a320-5dc5ee659903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:03.994797 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bdfec-ac13-42d3-a320-5dc5ee659903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:03.996435 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bdfec-ac13-42d3-a320-5dc5ee659903 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:04.007694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1069/2128] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:04.011880 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c256de-0842-4e08-b0cc-d5b28502f0c5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:04.024458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c256de-0842-4e08-b0cc-d5b28502f0c5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:04.030528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73707082-ff34-447f-87e3-31b6980a0bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:04.031477 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73707082-ff34-447f-87e3-31b6980a0bb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:04.031762 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73707082-ff34-447f-87e3-31b6980a0bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:04.032042 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73707082-ff34-447f-87e3-31b6980a0bb7 admin admin] PATH_INFO: `/v3/projects/12978f20d301496881b11bc3ba8bf5e2/users/3cc67573bb2b491b9c82b3657e5eed03/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:04.034166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73707082-ff34-447f-87e3-31b6980a0bb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12978f20d301496881b11bc3ba8bf5e2, user_id=3cc67573bb2b491b9c82b3657e5eed03, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:04.036909 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73707082-ff34-447f-87e3-31b6980a0bb7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:04.047282 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1060/2129] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:04 2022] PUT /identity/v3/projects/12978f20d301496881b11bc3ba8bf5e2/users/3cc67573bb2b491b9c82b3657e5eed03/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:04.058674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be158b-fc55-481b-b322-0a2ccce978e2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:04.058965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be158b-fc55-481b-b322-0a2ccce978e2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:04.059270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be158b-fc55-481b-b322-0a2ccce978e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:04.086148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-36be158b-fc55-481b-b322-0a2ccce978e2 None None] MFA Rules not processed for user `3cc67573bb2b491b9c82b3657e5eed03`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:04.087428 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-36be158b-fc55-481b-b322-0a2ccce978e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:04.114011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:04.162260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1070/2130] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:04.253866 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73707082-ff34-447f-87e3-31b6980a0bb7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:04.256011 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-73707082-ff34-447f-87e3-31b6980a0bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:04.317872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:04.348400 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73707082-ff34-447f-87e3-31b6980a0bb7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:04.352510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f55e9-f0d5-44a7-8466-5584adf15edf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:04.355206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f55e9-f0d5-44a7-8466-5584adf15edf service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:04.355206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f55e9-f0d5-44a7-8466-5584adf15edf service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:04.355206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f55e9-f0d5-44a7-8466-5584adf15edf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:04.422121 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f55e9-f0d5-44a7-8466-5584adf15edf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:04.424259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f55e9-f0d5-44a7-8466-5584adf15edf service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:04.435949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1061/2131] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:04.467097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:04.643840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:04.747062 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:05.026768 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:06.889535 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be158b-fc55-481b-b322-0a2ccce978e2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:06.899200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be158b-fc55-481b-b322-0a2ccce978e2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:06.902952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c80fd-7173-4b5d-b199-c29f93108123 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:06.908372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c80fd-7173-4b5d-b199-c29f93108123 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:06.908372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c80fd-7173-4b5d-b199-c29f93108123 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:06.908372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c80fd-7173-4b5d-b199-c29f93108123 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:06.969299 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c80fd-7173-4b5d-b199-c29f93108123 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:06.971723 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c80fd-7173-4b5d-b199-c29f93108123 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:06.975395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:06.981344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1071/2132] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:08.716505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f55e9-f0d5-44a7-8466-5584adf15edf service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:08.791770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f55e9-f0d5-44a7-8466-5584adf15edf service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:08.796997 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f16f6-e6a2-482f-8db4-5d7cbaecbd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:08.797962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f16f6-e6a2-482f-8db4-5d7cbaecbd90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:08.798316 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f16f6-e6a2-482f-8db4-5d7cbaecbd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:08.799300 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f16f6-e6a2-482f-8db4-5d7cbaecbd90 admin admin] PATH_INFO: `/v3/users/b3308f84d204454681b15647e0eaf059` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:08.800563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f16f6-e6a2-482f-8db4-5d7cbaecbd90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3308f84d204454681b15647e0eaf059)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:08.802658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f16f6-e6a2-482f-8db4-5d7cbaecbd90 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:09.006879 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-aa8f16f6-e6a2-482f-8db4-5d7cbaecbd90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3308f84d204454681b15647e0eaf059'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:09.022415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-aa8f16f6-e6a2-482f-8db4-5d7cbaecbd90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3308f84d204454681b15647e0eaf059'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:09.023408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-aa8f16f6-e6a2-482f-8db4-5d7cbaecbd90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3308f84d204454681b15647e0eaf059'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:09.033526 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-aa8f16f6-e6a2-482f-8db4-5d7cbaecbd90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3308f84d204454681b15647e0eaf059'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:09.034340 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-aa8f16f6-e6a2-482f-8db4-5d7cbaecbd90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3308f84d204454681b15647e0eaf059'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:09.145482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1062/2133] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:08 2022] DELETE /identity/v3/users/b3308f84d204454681b15647e0eaf059 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:09.150097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a35dbd-606f-45d5-8370-984f51202f00 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:09.150653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a35dbd-606f-45d5-8370-984f51202f00 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:09.150782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a35dbd-606f-45d5-8370-984f51202f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:09.168472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-05a35dbd-606f-45d5-8370-984f51202f00 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:09.169678 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-05a35dbd-606f-45d5-8370-984f51202f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:09.229102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1072/2134] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:09.239810 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f16f6-e6a2-482f-8db4-5d7cbaecbd90 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:09.241200 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-aa8f16f6-e6a2-482f-8db4-5d7cbaecbd90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:09.300151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f16f6-e6a2-482f-8db4-5d7cbaecbd90 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:09.303849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b089d8-c3d4-418d-8b93-400c253c9207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:09.304499 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b089d8-c3d4-418d-8b93-400c253c9207 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:09.304834 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b089d8-c3d4-418d-8b93-400c253c9207 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:09.305272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b089d8-c3d4-418d-8b93-400c253c9207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:09.314600 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b089d8-c3d4-418d-8b93-400c253c9207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:09.316146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b089d8-c3d4-418d-8b93-400c253c9207 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:09.325659 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1063/2135] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:09.609358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a35dbd-606f-45d5-8370-984f51202f00 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:09.610730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-05a35dbd-606f-45d5-8370-984f51202f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:09.670293 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a35dbd-606f-45d5-8370-984f51202f00 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:09.674112 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23f2a0-57ba-45c2-89d1-c0f5c8e3d7bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:09.674764 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23f2a0-57ba-45c2-89d1-c0f5c8e3d7bf service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:09.675046 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23f2a0-57ba-45c2-89d1-c0f5c8e3d7bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:09.675352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23f2a0-57ba-45c2-89d1-c0f5c8e3d7bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:09.677151 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-db23f2a0-57ba-45c2-89d1-c0f5c8e3d7bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:09.714194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b089d8-c3d4-418d-8b93-400c253c9207 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:09.715657 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-97b089d8-c3d4-418d-8b93-400c253c9207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:09.730330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b089d8-c3d4-418d-8b93-400c253c9207 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:09.754304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f8c45-8db8-4b36-97f3-4c4e9867fa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:09.758489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f8c45-8db8-4b36-97f3-4c4e9867fa3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:09.758489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f8c45-8db8-4b36-97f3-4c4e9867fa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:09.758489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f8c45-8db8-4b36-97f3-4c4e9867fa3e admin admin] PATH_INFO: `/v3/projects/301009c4e160494d97d1913a2ae3a0d7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:09.758489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f8c45-8db8-4b36-97f3-4c4e9867fa3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=301009c4e160494d97d1913a2ae3a0d7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:09.760607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f8c45-8db8-4b36-97f3-4c4e9867fa3e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:09.782329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:09.793062 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23f2a0-57ba-45c2-89d1-c0f5c8e3d7bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:09.795057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23f2a0-57ba-45c2-89d1-c0f5c8e3d7bf service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:09.816277 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1073/2136] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:09.873479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c11f8c45-8db8-4b36-97f3-4c4e9867fa3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '301009c4e160494d97d1913a2ae3a0d7', 'request_id': 'req-c11f8c45-8db8-4b36-97f3-4c4e9867fa3e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:09.878584 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c11f8c45-8db8-4b36-97f3-4c4e9867fa3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '301009c4e160494d97d1913a2ae3a0d7', 'request_id': 'req-c11f8c45-8db8-4b36-97f3-4c4e9867fa3e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:09.981239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c11f8c45-8db8-4b36-97f3-4c4e9867fa3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '301009c4e160494d97d1913a2ae3a0d7', 'request_id': 'req-c11f8c45-8db8-4b36-97f3-4c4e9867fa3e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:09.992409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c11f8c45-8db8-4b36-97f3-4c4e9867fa3e admin admin] The token cache is being invalidate because project 301009c4e160494d97d1913a2ae3a0d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:09.992694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c11f8c45-8db8-4b36-97f3-4c4e9867fa3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:09.993318 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c11f8c45-8db8-4b36-97f3-4c4e9867fa3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:09.994850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1064/2137] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:09 2022] DELETE /identity/v3/projects/301009c4e160494d97d1913a2ae3a0d7 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:10.000342 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05929d23-7837-4ca7-9f4c-fcda773069f1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:10.000635 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05929d23-7837-4ca7-9f4c-fcda773069f1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:10.001013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05929d23-7837-4ca7-9f4c-fcda773069f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:10.023045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-05929d23-7837-4ca7-9f4c-fcda773069f1 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:10.024284 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-05929d23-7837-4ca7-9f4c-fcda773069f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:10.081622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:10.085217 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1074/2138] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:10.093101 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871ae06-b5c8-48b7-a80c-9bc97558550d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:10.093388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871ae06-b5c8-48b7-a80c-9bc97558550d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:10.093838 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871ae06-b5c8-48b7-a80c-9bc97558550d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:10.114322 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0871ae06-b5c8-48b7-a80c-9bc97558550d None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:10.115546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0871ae06-b5c8-48b7-a80c-9bc97558550d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:10.123499 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1065/2139] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:10.128647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05929d23-7837-4ca7-9f4c-fcda773069f1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:10.139207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05929d23-7837-4ca7-9f4c-fcda773069f1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:10.144419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dcf2c7-d199-479c-baae-670033ac3618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:10.145101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcf2c7-d199-479c-baae-670033ac3618 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:10.145383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcf2c7-d199-479c-baae-670033ac3618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:10.145654 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcf2c7-d199-479c-baae-670033ac3618 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:10.146376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dcf2c7-d199-479c-baae-670033ac3618 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:10.146679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dcf2c7-d199-479c-baae-670033ac3618 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:10.148363 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dcf2c7-d199-479c-baae-670033ac3618 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:10.164502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1075/2140] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:20:10.165469 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871ae06-b5c8-48b7-a80c-9bc97558550d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:10.175393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871ae06-b5c8-48b7-a80c-9bc97558550d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:10.180211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fc626-2542-4836-a114-1ce634b73ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:10.180868 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fc626-2542-4836-a114-1ce634b73ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:10.181153 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fc626-2542-4836-a114-1ce634b73ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:10.181431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fc626-2542-4836-a114-1ce634b73ad8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:10.182119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048fc626-2542-4836-a114-1ce634b73ad8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:10.184076 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048fc626-2542-4836-a114-1ce634b73ad8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:10.286544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1066/2141] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:10 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:10.290731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dcf2c7-d199-479c-baae-670033ac3618 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:10.302128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dcf2c7-d199-479c-baae-670033ac3618 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:10.359271 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888789c8-ad0e-437c-a946-225fd5961415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:10.359873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888789c8-ad0e-437c-a946-225fd5961415 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:10.360152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888789c8-ad0e-437c-a946-225fd5961415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:10.360423 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888789c8-ad0e-437c-a946-225fd5961415 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:10.361670 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888789c8-ad0e-437c-a946-225fd5961415 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:10.363656 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888789c8-ad0e-437c-a946-225fd5961415 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:10.420587 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1076/2142] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:10 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:10.425483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fc626-2542-4836-a114-1ce634b73ad8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:10.435768 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fc626-2542-4836-a114-1ce634b73ad8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:10.441038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed902dd6-1ce8-42fc-905a-01046edd641b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:10.441695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed902dd6-1ce8-42fc-905a-01046edd641b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:10.441992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed902dd6-1ce8-42fc-905a-01046edd641b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:10.442271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed902dd6-1ce8-42fc-905a-01046edd641b admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:10.442984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed902dd6-1ce8-42fc-905a-01046edd641b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:10.444669 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed902dd6-1ce8-42fc-905a-01046edd641b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:10.455035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1067/2143] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:10.459125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888789c8-ad0e-437c-a946-225fd5961415 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:10.469441 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888789c8-ad0e-437c-a946-225fd5961415 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:10.474394 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbefcef-db9b-4e6d-a6a1-32ac0c11b69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:10.475092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbefcef-db9b-4e6d-a6a1-32ac0c11b69a admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:10.475439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbefcef-db9b-4e6d-a6a1-32ac0c11b69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:10.475721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbefcef-db9b-4e6d-a6a1-32ac0c11b69a admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:10.476451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbefcef-db9b-4e6d-a6a1-32ac0c11b69a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:10.478339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbefcef-db9b-4e6d-a6a1-32ac0c11b69a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:10.489225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1077/2144] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:10.493437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed902dd6-1ce8-42fc-905a-01046edd641b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:10.504463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed902dd6-1ce8-42fc-905a-01046edd641b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:10.509133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff3b24-669b-4630-81ea-01a4ff9a80bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:10.510016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff3b24-669b-4630-81ea-01a4ff9a80bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:10.510299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff3b24-669b-4630-81ea-01a4ff9a80bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:10.510602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff3b24-669b-4630-81ea-01a4ff9a80bb admin admin] PATH_INFO: `/v3/projects/c46d496716274a2ea1108b37113a3d00/users/6c85daa534624e53a0e6a589d8e68f0a/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:10.525841 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff3b24-669b-4630-81ea-01a4ff9a80bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46d496716274a2ea1108b37113a3d00, user_id=6c85daa534624e53a0e6a589d8e68f0a, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:10.528594 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff3b24-669b-4630-81ea-01a4ff9a80bb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:10.609587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1068/2145] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:10 2022] PUT /identity/v3/projects/c46d496716274a2ea1108b37113a3d00/users/6c85daa534624e53a0e6a589d8e68f0a/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:10.620314 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c6898-6f46-4fc2-8f42-a3a236b2f7ff None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:10.620604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c6898-6f46-4fc2-8f42-a3a236b2f7ff None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:10.620886 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c6898-6f46-4fc2-8f42-a3a236b2f7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:10.647811 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ed3c6898-6f46-4fc2-8f42-a3a236b2f7ff None None] MFA Rules not processed for user `6c85daa534624e53a0e6a589d8e68f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:10.649007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ed3c6898-6f46-4fc2-8f42-a3a236b2f7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:10.710666 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1078/2146] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:10.722152 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff3b24-669b-4630-81ea-01a4ff9a80bb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:10.723801 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dbff3b24-669b-4630-81ea-01a4ff9a80bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:10.800494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff3b24-669b-4630-81ea-01a4ff9a80bb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:10.807491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5cb92-cfc8-432a-a5c9-5d769e8f9a02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:10.807491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5cb92-cfc8-432a-a5c9-5d769e8f9a02 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:10.807491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5cb92-cfc8-432a-a5c9-5d769e8f9a02 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:10.807491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5cb92-cfc8-432a-a5c9-5d769e8f9a02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:10.818741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5cb92-cfc8-432a-a5c9-5d769e8f9a02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:10.820514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5cb92-cfc8-432a-a5c9-5d769e8f9a02 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:10.838172 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1069/2147] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:12.230733 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c6898-6f46-4fc2-8f42-a3a236b2f7ff None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:12.285544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:12.302359 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c6898-6f46-4fc2-8f42-a3a236b2f7ff None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:12.306947 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1822029-d123-409b-88e6-840ed154637d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:12.307858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1822029-d123-409b-88e6-840ed154637d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:12.308144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1822029-d123-409b-88e6-840ed154637d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:12.308423 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1822029-d123-409b-88e6-840ed154637d admin admin] PATH_INFO: `/v3/users/6c85daa534624e53a0e6a589d8e68f0a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:12.309625 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1822029-d123-409b-88e6-840ed154637d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c85daa534624e53a0e6a589d8e68f0a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:12.311663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1822029-d123-409b-88e6-840ed154637d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:12.575066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b1822029-d123-409b-88e6-840ed154637d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c85daa534624e53a0e6a589d8e68f0a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:12.587608 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b1822029-d123-409b-88e6-840ed154637d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c85daa534624e53a0e6a589d8e68f0a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:12.649098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b1822029-d123-409b-88e6-840ed154637d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c85daa534624e53a0e6a589d8e68f0a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:12.649725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b1822029-d123-409b-88e6-840ed154637d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c85daa534624e53a0e6a589d8e68f0a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:12.657493 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b1822029-d123-409b-88e6-840ed154637d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c85daa534624e53a0e6a589d8e68f0a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:12.664215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1079/2148] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:12 2022] DELETE /identity/v3/users/6c85daa534624e53a0e6a589d8e68f0a => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:12.668844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59be797-e140-4716-93f6-c492bb831b71 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:12.669331 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59be797-e140-4716-93f6-c492bb831b71 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:12.669421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59be797-e140-4716-93f6-c492bb831b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:12.686638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f59be797-e140-4716-93f6-c492bb831b71 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:12.687934 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f59be797-e140-4716-93f6-c492bb831b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:12.700132 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:12.747853 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1070/2149] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:12.757700 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1822029-d123-409b-88e6-840ed154637d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:12.759120 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b1822029-d123-409b-88e6-840ed154637d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:12.825663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1822029-d123-409b-88e6-840ed154637d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:12.826372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61d954-9775-46ef-a663-9a10b640c377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:12.827051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61d954-9775-46ef-a663-9a10b640c377 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:12.827361 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61d954-9775-46ef-a663-9a10b640c377 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:12.827638 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61d954-9775-46ef-a663-9a10b640c377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:12.837567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61d954-9775-46ef-a663-9a10b640c377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:12.840254 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61d954-9775-46ef-a663-9a10b640c377 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:12.851404 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1080/2150] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:13.251395 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59be797-e140-4716-93f6-c492bb831b71 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:13.253159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f59be797-e140-4716-93f6-c492bb831b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:13.265864 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59be797-e140-4716-93f6-c492bb831b71 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:13.271004 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f25db-5cfe-4390-adaa-4fdc9d98867d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:13.274145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f25db-5cfe-4390-adaa-4fdc9d98867d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:13.274145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f25db-5cfe-4390-adaa-4fdc9d98867d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:13.274145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f25db-5cfe-4390-adaa-4fdc9d98867d admin admin] PATH_INFO: `/v3/projects/c46d496716274a2ea1108b37113a3d00` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:13.274145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f25db-5cfe-4390-adaa-4fdc9d98867d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c46d496716274a2ea1108b37113a3d00)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:13.276047 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f25db-5cfe-4390-adaa-4fdc9d98867d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:13.410126 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5d1f25db-5cfe-4390-adaa-4fdc9d98867d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c46d496716274a2ea1108b37113a3d00', 'request_id': 'req-5d1f25db-5cfe-4390-adaa-4fdc9d98867d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:13.417020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5d1f25db-5cfe-4390-adaa-4fdc9d98867d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c46d496716274a2ea1108b37113a3d00', 'request_id': 'req-5d1f25db-5cfe-4390-adaa-4fdc9d98867d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:13.518724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5d1f25db-5cfe-4390-adaa-4fdc9d98867d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c46d496716274a2ea1108b37113a3d00', 'request_id': 'req-5d1f25db-5cfe-4390-adaa-4fdc9d98867d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:13.530019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5d1f25db-5cfe-4390-adaa-4fdc9d98867d admin admin] The token cache is being invalidate because project c46d496716274a2ea1108b37113a3d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:13.530306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5d1f25db-5cfe-4390-adaa-4fdc9d98867d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:13.530935 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5d1f25db-5cfe-4390-adaa-4fdc9d98867d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:13.532499 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1071/2151] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:13 2022] DELETE /identity/v3/projects/c46d496716274a2ea1108b37113a3d00 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:13.537907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47122e-26af-47a3-ab59-9228abba5b83 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:13.538407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47122e-26af-47a3-ab59-9228abba5b83 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:13.538570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47122e-26af-47a3-ab59-9228abba5b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:13.556805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3f47122e-26af-47a3-ab59-9228abba5b83 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:13.557910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3f47122e-26af-47a3-ab59-9228abba5b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:13.615138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1081/2152] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:13.624010 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1daf47-c435-40a3-a448-cf9913799ea7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:13.624298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1daf47-c435-40a3-a448-cf9913799ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:13.624579 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1daf47-c435-40a3-a448-cf9913799ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:13.642765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ce1daf47-c435-40a3-a448-cf9913799ea7 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:13.643982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ce1daf47-c435-40a3-a448-cf9913799ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:13.651769 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1072/2153] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:13.656086 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47122e-26af-47a3-ab59-9228abba5b83 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:13.666488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47122e-26af-47a3-ab59-9228abba5b83 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:13.670996 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91725537-bbc2-4d49-ac30-02fd6bcc83e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:13.671679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91725537-bbc2-4d49-ac30-02fd6bcc83e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:13.671962 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91725537-bbc2-4d49-ac30-02fd6bcc83e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:13.672235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91725537-bbc2-4d49-ac30-02fd6bcc83e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:13.672981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91725537-bbc2-4d49-ac30-02fd6bcc83e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:13.673266 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91725537-bbc2-4d49-ac30-02fd6bcc83e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:13.674843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91725537-bbc2-4d49-ac30-02fd6bcc83e1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:13.676327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:13.686491 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1082/2154] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:20:13.690878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1daf47-c435-40a3-a448-cf9913799ea7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:13.700088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1daf47-c435-40a3-a448-cf9913799ea7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:13.704941 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23f9ec-d0c2-41af-a4a3-c23a53529471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:13.705777 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23f9ec-d0c2-41af-a4a3-c23a53529471 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:13.706063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23f9ec-d0c2-41af-a4a3-c23a53529471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:13.706346 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23f9ec-d0c2-41af-a4a3-c23a53529471 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:13.707064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23f9ec-d0c2-41af-a4a3-c23a53529471 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:13.709096 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23f9ec-d0c2-41af-a4a3-c23a53529471 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:13.816821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1073/2155] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:13 2022] POST /identity/v3/projects => generated 405 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:13.820757 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91725537-bbc2-4d49-ac30-02fd6bcc83e1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:13.845840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91725537-bbc2-4d49-ac30-02fd6bcc83e1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:13.891567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e2234-9ee7-4aa6-b384-7ff8a5eb7545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:13.892242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e2234-9ee7-4aa6-b384-7ff8a5eb7545 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:13.892544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e2234-9ee7-4aa6-b384-7ff8a5eb7545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:13.892969 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e2234-9ee7-4aa6-b384-7ff8a5eb7545 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:13.893672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e2234-9ee7-4aa6-b384-7ff8a5eb7545 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:13.895643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e2234-9ee7-4aa6-b384-7ff8a5eb7545 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:14.009544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1083/2156] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:13 2022] POST /identity/v3/users => generated 303 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:14.013877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23f9ec-d0c2-41af-a4a3-c23a53529471 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:14.024274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23f9ec-d0c2-41af-a4a3-c23a53529471 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:14.028833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52230e0c-8350-4170-bad8-ce4448f1204c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:14.029851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52230e0c-8350-4170-bad8-ce4448f1204c admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:14.030134 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52230e0c-8350-4170-bad8-ce4448f1204c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:14.030405 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52230e0c-8350-4170-bad8-ce4448f1204c admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:14.031123 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52230e0c-8350-4170-bad8-ce4448f1204c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:14.033295 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52230e0c-8350-4170-bad8-ce4448f1204c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:14.044215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1074/2157] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:14.047933 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e2234-9ee7-4aa6-b384-7ff8a5eb7545 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:14.058305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e2234-9ee7-4aa6-b384-7ff8a5eb7545 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:14.062900 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d2b57-5e3a-4065-8e96-4f052f6894c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:14.063584 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d2b57-5e3a-4065-8e96-4f052f6894c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:14.063870 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d2b57-5e3a-4065-8e96-4f052f6894c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:14.064204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d2b57-5e3a-4065-8e96-4f052f6894c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:14.064923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d2b57-5e3a-4065-8e96-4f052f6894c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:14.066511 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d2b57-5e3a-4065-8e96-4f052f6894c8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:14.076519 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1084/2158] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:14.080900 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52230e0c-8350-4170-bad8-ce4448f1204c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:14.090802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52230e0c-8350-4170-bad8-ce4448f1204c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:14.096287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658c780-ae80-49c8-8831-f5cf1deebd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:14.101746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658c780-ae80-49c8-8831-f5cf1deebd1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:14.101746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658c780-ae80-49c8-8831-f5cf1deebd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:14.101746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658c780-ae80-49c8-8831-f5cf1deebd1a admin admin] PATH_INFO: `/v3/projects/c1c44dd08a0b483ab9fff2d360593dda/users/bba6609ae2c84986a939f69c26af0ce1/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:14.112960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658c780-ae80-49c8-8831-f5cf1deebd1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c44dd08a0b483ab9fff2d360593dda, user_id=bba6609ae2c84986a939f69c26af0ce1, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:14.115634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658c780-ae80-49c8-8831-f5cf1deebd1a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:14.186647 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1075/2159] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:14 2022] PUT /identity/v3/projects/c1c44dd08a0b483ab9fff2d360593dda/users/bba6609ae2c84986a939f69c26af0ce1/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:14.197464 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8729d0-e6f5-4890-807a-11746b788439 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:14.197784 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8729d0-e6f5-4890-807a-11746b788439 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:14.198092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8729d0-e6f5-4890-807a-11746b788439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:14.226209 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1e8729d0-e6f5-4890-807a-11746b788439 None None] MFA Rules not processed for user `bba6609ae2c84986a939f69c26af0ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:14.227454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1e8729d0-e6f5-4890-807a-11746b788439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:14.293791 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1085/2160] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:14 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:14.298244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658c780-ae80-49c8-8831-f5cf1deebd1a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:14.338979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:14.359829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658c780-ae80-49c8-8831-f5cf1deebd1a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:14.365154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3eaaa-3f78-4228-a982-0b36a834b6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:14.365868 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3eaaa-3f78-4228-a982-0b36a834b6fc admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:14.366178 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3eaaa-3f78-4228-a982-0b36a834b6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:14.366455 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3eaaa-3f78-4228-a982-0b36a834b6fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:14.367151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3eaaa-3f78-4228-a982-0b36a834b6fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:14.369163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3eaaa-3f78-4228-a982-0b36a834b6fc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:14.490550 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1076/2161] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:14 2022] POST /identity/v3/projects => generated 405 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:14.494480 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8729d0-e6f5-4890-807a-11746b788439 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:14.505181 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8729d0-e6f5-4890-807a-11746b788439 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:14.567144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1368ea4-634c-4608-9bab-143df6bf80ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:14.567911 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1368ea4-634c-4608-9bab-143df6bf80ea admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:14.568196 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1368ea4-634c-4608-9bab-143df6bf80ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:14.568501 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1368ea4-634c-4608-9bab-143df6bf80ea admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:14.569186 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1368ea4-634c-4608-9bab-143df6bf80ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:14.571139 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1368ea4-634c-4608-9bab-143df6bf80ea admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:14.578528 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:14.706492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1086/2162] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:14 2022] POST /identity/v3/users => generated 309 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:14.710516 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3eaaa-3f78-4228-a982-0b36a834b6fc admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:14.721094 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3eaaa-3f78-4228-a982-0b36a834b6fc admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:14.725745 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620420e5-0434-46f6-abe2-09327c6abc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:14.726399 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620420e5-0434-46f6-abe2-09327c6abc6f admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:14.726695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620420e5-0434-46f6-abe2-09327c6abc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:14.726956 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620420e5-0434-46f6-abe2-09327c6abc6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:14.727765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620420e5-0434-46f6-abe2-09327c6abc6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:14.729340 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620420e5-0434-46f6-abe2-09327c6abc6f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:14.740726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1077/2163] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:14.744737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1368ea4-634c-4608-9bab-143df6bf80ea admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:14.754602 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1368ea4-634c-4608-9bab-143df6bf80ea admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:14.759118 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18761b24-24fd-4ff7-89eb-acf51aeaf082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:14.760013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18761b24-24fd-4ff7-89eb-acf51aeaf082 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:14.760290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18761b24-24fd-4ff7-89eb-acf51aeaf082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:14.760561 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18761b24-24fd-4ff7-89eb-acf51aeaf082 admin admin] PATH_INFO: `/v3/projects/1e9c90c942b44f53b1df35478b4cfaf3/users/85e79d8d971245b4a3ae935345e00727/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:14.774710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18761b24-24fd-4ff7-89eb-acf51aeaf082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9c90c942b44f53b1df35478b4cfaf3, user_id=85e79d8d971245b4a3ae935345e00727, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:14.777350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18761b24-24fd-4ff7-89eb-acf51aeaf082 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:14.849677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1087/2164] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:14 2022] PUT /identity/v3/projects/1e9c90c942b44f53b1df35478b4cfaf3/users/85e79d8d971245b4a3ae935345e00727/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:14.853729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620420e5-0434-46f6-abe2-09327c6abc6f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:14.863900 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620420e5-0434-46f6-abe2-09327c6abc6f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:14.922773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511c15d-26d3-45df-9f00-e505564ef6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:14.923682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511c15d-26d3-45df-9f00-e505564ef6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:14.924011 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511c15d-26d3-45df-9f00-e505564ef6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:14.924333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511c15d-26d3-45df-9f00-e505564ef6c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:14.925072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511c15d-26d3-45df-9f00-e505564ef6c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:14.926674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511c15d-26d3-45df-9f00-e505564ef6c6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:14.938252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1078/2165] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:14.942221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18761b24-24fd-4ff7-89eb-acf51aeaf082 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:14.952613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18761b24-24fd-4ff7-89eb-acf51aeaf082 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:14.965844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26534286-909f-4bbd-9662-eb8355d1f563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:14.966841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26534286-909f-4bbd-9662-eb8355d1f563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:14.967205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26534286-909f-4bbd-9662-eb8355d1f563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:14.967602 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26534286-909f-4bbd-9662-eb8355d1f563 admin admin] PATH_INFO: `/v3/projects/1e9c90c942b44f53b1df35478b4cfaf3/users/85e79d8d971245b4a3ae935345e00727/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:14.969617 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26534286-909f-4bbd-9662-eb8355d1f563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9c90c942b44f53b1df35478b4cfaf3, user_id=85e79d8d971245b4a3ae935345e00727, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:14.973231 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26534286-909f-4bbd-9662-eb8355d1f563 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:14.984590 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1088/2166] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:14 2022] PUT /identity/v3/projects/1e9c90c942b44f53b1df35478b4cfaf3/users/85e79d8d971245b4a3ae935345e00727/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:14.995574 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bd361-6750-4d05-b7a6-7a3604ae25ae None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:14.995935 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bd361-6750-4d05-b7a6-7a3604ae25ae None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:14.996277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bd361-6750-4d05-b7a6-7a3604ae25ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:15.023376 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-713bd361-6750-4d05-b7a6-7a3604ae25ae None None] MFA Rules not processed for user `85e79d8d971245b4a3ae935345e00727`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:15.024557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-713bd361-6750-4d05-b7a6-7a3604ae25ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:15.093730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1079/2167] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:14 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:15.106595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26534286-909f-4bbd-9662-eb8355d1f563 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:15.108440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-26534286-909f-4bbd-9662-eb8355d1f563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:15.170721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26534286-909f-4bbd-9662-eb8355d1f563 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:15.175302 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c121616-5390-4ca5-941e-e90f365426a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:15.175954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c121616-5390-4ca5-941e-e90f365426a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:15.176236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c121616-5390-4ca5-941e-e90f365426a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:15.176543 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c121616-5390-4ca5-941e-e90f365426a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:15.185800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c121616-5390-4ca5-941e-e90f365426a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:15.187262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c121616-5390-4ca5-941e-e90f365426a3 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:15.197879 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1089/2168] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:15 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:15.338393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bd361-6750-4d05-b7a6-7a3604ae25ae None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:15.373441 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:15.403693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bd361-6750-4d05-b7a6-7a3604ae25ae None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:15.409061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c941c-8d19-423f-a0c3-3bca6ec54bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:15.410032 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c941c-8d19-423f-a0c3-3bca6ec54bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:15.410250 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c941c-8d19-423f-a0c3-3bca6ec54bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:15.410553 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c941c-8d19-423f-a0c3-3bca6ec54bcb admin admin] PATH_INFO: `/v3/users/bba6609ae2c84986a939f69c26af0ce1` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:15.416758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c941c-8d19-423f-a0c3-3bca6ec54bcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bba6609ae2c84986a939f69c26af0ce1)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:15.418807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c941c-8d19-423f-a0c3-3bca6ec54bcb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:15.655624 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-912c941c-8d19-423f-a0c3-3bca6ec54bcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bba6609ae2c84986a939f69c26af0ce1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:15.669140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-912c941c-8d19-423f-a0c3-3bca6ec54bcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bba6609ae2c84986a939f69c26af0ce1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:15.669932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-912c941c-8d19-423f-a0c3-3bca6ec54bcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bba6609ae2c84986a939f69c26af0ce1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:15.677871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-912c941c-8d19-423f-a0c3-3bca6ec54bcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bba6609ae2c84986a939f69c26af0ce1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:15.678543 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-912c941c-8d19-423f-a0c3-3bca6ec54bcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bba6609ae2c84986a939f69c26af0ce1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:15.766159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1080/2169] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:15 2022] DELETE /identity/v3/users/bba6609ae2c84986a939f69c26af0ce1 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:15.770825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a3d35-c146-4773-8516-3f3a166c90ec None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:15.771343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a3d35-c146-4773-8516-3f3a166c90ec None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:15.771550 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a3d35-c146-4773-8516-3f3a166c90ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:15.789237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a45a3d35-c146-4773-8516-3f3a166c90ec None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:15.790457 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a45a3d35-c146-4773-8516-3f3a166c90ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:15.848314 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1090/2170] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:15.858655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c941c-8d19-423f-a0c3-3bca6ec54bcb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:15.860139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-912c941c-8d19-423f-a0c3-3bca6ec54bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:15.921057 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c941c-8d19-423f-a0c3-3bca6ec54bcb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:15.925644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4c891-f7fc-4d05-857e-15ab1f86e83f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:15.926375 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4c891-f7fc-4d05-857e-15ab1f86e83f service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:15.926665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4c891-f7fc-4d05-857e-15ab1f86e83f service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:15.926970 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4c891-f7fc-4d05-857e-15ab1f86e83f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:15.937160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4c891-f7fc-4d05-857e-15ab1f86e83f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:15.938952 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4c891-f7fc-4d05-857e-15ab1f86e83f service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:15.950125 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1081/2171] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:16.565524 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a3d35-c146-4773-8516-3f3a166c90ec None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:16.567000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a45a3d35-c146-4773-8516-3f3a166c90ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:16.598565 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4c891-f7fc-4d05-857e-15ab1f86e83f service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:16.600444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a4c4c891-f7fc-4d05-857e-15ab1f86e83f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:16.615930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4c891-f7fc-4d05-857e-15ab1f86e83f service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:16.621576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86024149-f7f6-40eb-8b54-d3e7a2152a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:16.622527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86024149-f7f6-40eb-8b54-d3e7a2152a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:16.622819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86024149-f7f6-40eb-8b54-d3e7a2152a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:16.623291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86024149-f7f6-40eb-8b54-d3e7a2152a70 admin admin] PATH_INFO: `/v3/projects/c1c44dd08a0b483ab9fff2d360593dda` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:16.624907 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86024149-f7f6-40eb-8b54-d3e7a2152a70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c44dd08a0b483ab9fff2d360593dda)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:16.628398 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86024149-f7f6-40eb-8b54-d3e7a2152a70 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:16.653550 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a3d35-c146-4773-8516-3f3a166c90ec None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:16.660450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6382a2-2baa-4a55-988f-64a0c931aa31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:16.661245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6382a2-2baa-4a55-988f-64a0c931aa31 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:16.661646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6382a2-2baa-4a55-988f-64a0c931aa31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:16.662058 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6382a2-2baa-4a55-988f-64a0c931aa31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:16.664101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cd6382a2-2baa-4a55-988f-64a0c931aa31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:16.677142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:16.730004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6382a2-2baa-4a55-988f-64a0c931aa31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:16.731469 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6382a2-2baa-4a55-988f-64a0c931aa31 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:16.741142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1091/2172] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:16.780112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-86024149-f7f6-40eb-8b54-d3e7a2152a70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c44dd08a0b483ab9fff2d360593dda', 'request_id': 'req-86024149-f7f6-40eb-8b54-d3e7a2152a70'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:16.782147 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:16.790805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-86024149-f7f6-40eb-8b54-d3e7a2152a70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c44dd08a0b483ab9fff2d360593dda', 'request_id': 'req-86024149-f7f6-40eb-8b54-d3e7a2152a70'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:16.872419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-86024149-f7f6-40eb-8b54-d3e7a2152a70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c44dd08a0b483ab9fff2d360593dda', 'request_id': 'req-86024149-f7f6-40eb-8b54-d3e7a2152a70'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:16.890078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-86024149-f7f6-40eb-8b54-d3e7a2152a70 admin admin] The token cache is being invalidate because project c1c44dd08a0b483ab9fff2d360593dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:16.890543 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-86024149-f7f6-40eb-8b54-d3e7a2152a70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:16.891353 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-86024149-f7f6-40eb-8b54-d3e7a2152a70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:16.896844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1082/2173] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:16 2022] DELETE /identity/v3/projects/c1c44dd08a0b483ab9fff2d360593dda => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:16.898162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6382a2-2baa-4a55-988f-64a0c931aa31 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:16.913449 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6382a2-2baa-4a55-988f-64a0c931aa31 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:16.974239 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8c724-18ed-444b-a153-941a70d95714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:16.975118 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8c724-18ed-444b-a153-941a70d95714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:16.975460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8c724-18ed-444b-a153-941a70d95714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:16.975766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8c724-18ed-444b-a153-941a70d95714 admin admin] PATH_INFO: `/v3/users/85e79d8d971245b4a3ae935345e00727` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:16.976962 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8c724-18ed-444b-a153-941a70d95714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85e79d8d971245b4a3ae935345e00727)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:16.979068 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8c724-18ed-444b-a153-941a70d95714 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:17.234874 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-66f8c724-18ed-444b-a153-941a70d95714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85e79d8d971245b4a3ae935345e00727'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:17.246840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-66f8c724-18ed-444b-a153-941a70d95714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85e79d8d971245b4a3ae935345e00727'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:17.322280 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-66f8c724-18ed-444b-a153-941a70d95714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85e79d8d971245b4a3ae935345e00727'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:17.322902 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-66f8c724-18ed-444b-a153-941a70d95714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85e79d8d971245b4a3ae935345e00727'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:17.332407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-66f8c724-18ed-444b-a153-941a70d95714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85e79d8d971245b4a3ae935345e00727'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:17.339347 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1092/2174] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:16 2022] DELETE /identity/v3/users/85e79d8d971245b4a3ae935345e00727 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:17.823823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86024149-f7f6-40eb-8b54-d3e7a2152a70 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:17.825450 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-86024149-f7f6-40eb-8b54-d3e7a2152a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:17.849759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86024149-f7f6-40eb-8b54-d3e7a2152a70 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:17.913380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca70a1-f777-4446-b046-81a9991aa54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:17.914336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca70a1-f777-4446-b046-81a9991aa54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:17.914628 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca70a1-f777-4446-b046-81a9991aa54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:17.914917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca70a1-f777-4446-b046-81a9991aa54e admin admin] PATH_INFO: `/v3/projects/1e9c90c942b44f53b1df35478b4cfaf3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:17.916382 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca70a1-f777-4446-b046-81a9991aa54e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e9c90c942b44f53b1df35478b4cfaf3)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:17.918405 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca70a1-f777-4446-b046-81a9991aa54e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:18.065139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-45ca70a1-f777-4446-b046-81a9991aa54e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e9c90c942b44f53b1df35478b4cfaf3', 'request_id': 'req-45ca70a1-f777-4446-b046-81a9991aa54e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:18.072525 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-45ca70a1-f777-4446-b046-81a9991aa54e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e9c90c942b44f53b1df35478b4cfaf3', 'request_id': 'req-45ca70a1-f777-4446-b046-81a9991aa54e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:18.135061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-45ca70a1-f777-4446-b046-81a9991aa54e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e9c90c942b44f53b1df35478b4cfaf3', 'request_id': 'req-45ca70a1-f777-4446-b046-81a9991aa54e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:18.146906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-45ca70a1-f777-4446-b046-81a9991aa54e admin admin] The token cache is being invalidate because project 1e9c90c942b44f53b1df35478b4cfaf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:18.147311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-45ca70a1-f777-4446-b046-81a9991aa54e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:18.148022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-45ca70a1-f777-4446-b046-81a9991aa54e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:18.149659 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1083/2175] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:17 2022] DELETE /identity/v3/projects/1e9c90c942b44f53b1df35478b4cfaf3 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:18.154971 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83454732-22da-4f90-88e1-bba14718d926 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:18.155474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83454732-22da-4f90-88e1-bba14718d926 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:18.155565 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83454732-22da-4f90-88e1-bba14718d926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:18.179957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-83454732-22da-4f90-88e1-bba14718d926 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:18.179957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-83454732-22da-4f90-88e1-bba14718d926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:18.224221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:18.228533 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca70a1-f777-4446-b046-81a9991aa54e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:18.230438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-45ca70a1-f777-4446-b046-81a9991aa54e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:18.264692 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1093/2176] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:18.272892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5feab93-cfd9-4782-9328-0646551733c6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:18.275783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5feab93-cfd9-4782-9328-0646551733c6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:18.276343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5feab93-cfd9-4782-9328-0646551733c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:18.300554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b5feab93-cfd9-4782-9328-0646551733c6 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:18.301924 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b5feab93-cfd9-4782-9328-0646551733c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:18.311606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1094/2177] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:18.316344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5feab93-cfd9-4782-9328-0646551733c6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:18.324396 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca70a1-f777-4446-b046-81a9991aa54e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:18.327310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5feab93-cfd9-4782-9328-0646551733c6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:18.330976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb83771-bd08-4cc2-b889-7ee9ac88d79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:18.331915 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb83771-bd08-4cc2-b889-7ee9ac88d79e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:18.332793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb83771-bd08-4cc2-b889-7ee9ac88d79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:18.332793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb83771-bd08-4cc2-b889-7ee9ac88d79e admin admin] PATH_INFO: `/v3/users/fad6c305398f44299afa208a2aa36711` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:18.333711 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d8d45-10dc-43a0-b490-09fb3909e808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:18.333843 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb83771-bd08-4cc2-b889-7ee9ac88d79e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fad6c305398f44299afa208a2aa36711)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:18.334689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d8d45-10dc-43a0-b490-09fb3909e808 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:18.335278 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d8d45-10dc-43a0-b490-09fb3909e808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:18.335769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d8d45-10dc-43a0-b490-09fb3909e808 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:18.335898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb83771-bd08-4cc2-b889-7ee9ac88d79e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:18.336848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d8d45-10dc-43a0-b490-09fb3909e808 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:18.337544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d8d45-10dc-43a0-b490-09fb3909e808 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:18.339683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d8d45-10dc-43a0-b490-09fb3909e808 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:18.350913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:18.357904 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1095/2178] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:20:18.363634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d8d45-10dc-43a0-b490-09fb3909e808 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:18.373362 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d8d45-10dc-43a0-b490-09fb3909e808 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:18.378334 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b708163-22eb-41a9-8e94-43e889322973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:18.378999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b708163-22eb-41a9-8e94-43e889322973 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:18.379318 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b708163-22eb-41a9-8e94-43e889322973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:18.379596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b708163-22eb-41a9-8e94-43e889322973 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:18.380317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b708163-22eb-41a9-8e94-43e889322973 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:18.382202 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b708163-22eb-41a9-8e94-43e889322973 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:18.502565 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1096/2179] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:18 2022] POST /identity/v3/projects => generated 393 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:18.503710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b708163-22eb-41a9-8e94-43e889322973 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:18.505295 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7b708163-22eb-41a9-8e94-43e889322973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:18.539590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bcb83771-bd08-4cc2-b889-7ee9ac88d79e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fad6c305398f44299afa208a2aa36711'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:18.553796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bcb83771-bd08-4cc2-b889-7ee9ac88d79e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fad6c305398f44299afa208a2aa36711'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:18.554734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bcb83771-bd08-4cc2-b889-7ee9ac88d79e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fad6c305398f44299afa208a2aa36711'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:18.562025 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bcb83771-bd08-4cc2-b889-7ee9ac88d79e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fad6c305398f44299afa208a2aa36711'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:18.562795 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bcb83771-bd08-4cc2-b889-7ee9ac88d79e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fad6c305398f44299afa208a2aa36711'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:18.586463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b708163-22eb-41a9-8e94-43e889322973 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:18.591661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7241d2-bdef-48ed-950d-f570d9b3559e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:18.592568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7241d2-bdef-48ed-950d-f570d9b3559e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:18.592904 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7241d2-bdef-48ed-950d-f570d9b3559e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:18.593186 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7241d2-bdef-48ed-950d-f570d9b3559e admin admin] PATH_INFO: `/v3/users/52e999e636af44b79e9f64aff5b5294a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:18.594439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7241d2-bdef-48ed-950d-f570d9b3559e admin admin] RBAC: Authorizing `identity:delete_user(user_id=52e999e636af44b79e9f64aff5b5294a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:18.597214 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7241d2-bdef-48ed-950d-f570d9b3559e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:18.674385 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1084/2180] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:18 2022] DELETE /identity/v3/users/fad6c305398f44299afa208a2aa36711 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:18.674385 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb83771-bd08-4cc2-b889-7ee9ac88d79e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:18.674597 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bcb83771-bd08-4cc2-b889-7ee9ac88d79e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:18.749588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb83771-bd08-4cc2-b889-7ee9ac88d79e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:18.754856 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13583e-dfd4-477b-b8c5-c92ed1d4efd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:18.755564 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13583e-dfd4-477b-b8c5-c92ed1d4efd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:18.755850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13583e-dfd4-477b-b8c5-c92ed1d4efd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:18.756124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13583e-dfd4-477b-b8c5-c92ed1d4efd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:18.756820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13583e-dfd4-477b-b8c5-c92ed1d4efd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:18.758740 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13583e-dfd4-477b-b8c5-c92ed1d4efd3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:18.799713 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa7241d2-bdef-48ed-950d-f570d9b3559e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52e999e636af44b79e9f64aff5b5294a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:18.813679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa7241d2-bdef-48ed-950d-f570d9b3559e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52e999e636af44b79e9f64aff5b5294a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:18.833237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1085/2181] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:18 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:18.835060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8721008-b099-4ed1-8393-40e7d091ec24 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:18.835503 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8721008-b099-4ed1-8393-40e7d091ec24 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:18.836079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8721008-b099-4ed1-8393-40e7d091ec24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:18.854918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a8721008-b099-4ed1-8393-40e7d091ec24 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:18.856256 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a8721008-b099-4ed1-8393-40e7d091ec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:18.867663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1086/2182] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:18.867663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8721008-b099-4ed1-8393-40e7d091ec24 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:18.875298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8721008-b099-4ed1-8393-40e7d091ec24 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:18.878187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa7241d2-bdef-48ed-950d-f570d9b3559e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52e999e636af44b79e9f64aff5b5294a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:18.878314 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa7241d2-bdef-48ed-950d-f570d9b3559e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52e999e636af44b79e9f64aff5b5294a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:18.886615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa7241d2-bdef-48ed-950d-f570d9b3559e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52e999e636af44b79e9f64aff5b5294a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:18.893257 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3d026-1344-49b5-b6de-a82c36d71991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:18.893859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3d026-1344-49b5-b6de-a82c36d71991 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:18.894142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3d026-1344-49b5-b6de-a82c36d71991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:18.894422 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3d026-1344-49b5-b6de-a82c36d71991 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:18.895210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3d026-1344-49b5-b6de-a82c36d71991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:18.896784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3d026-1344-49b5-b6de-a82c36d71991 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:18.902477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1097/2183] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:18 2022] DELETE /identity/v3/users/52e999e636af44b79e9f64aff5b5294a => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:18.902477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7241d2-bdef-48ed-950d-f570d9b3559e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:18.903493 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-aa7241d2-bdef-48ed-950d-f570d9b3559e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:18.914946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:18.918420 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1087/2184] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:18.923090 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aea2b-2628-4e81-b0b2-129771fdd26f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:18.923431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aea2b-2628-4e81-b0b2-129771fdd26f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:18.925143 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aea2b-2628-4e81-b0b2-129771fdd26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:18.951083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c87aea2b-2628-4e81-b0b2-129771fdd26f None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:18.952389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c87aea2b-2628-4e81-b0b2-129771fdd26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:18.991710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7241d2-bdef-48ed-950d-f570d9b3559e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:18.995580 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256b309-9bf8-44db-b02c-86746475601b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:18.996237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256b309-9bf8-44db-b02c-86746475601b service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:18.996521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256b309-9bf8-44db-b02c-86746475601b service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:18.996801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256b309-9bf8-44db-b02c-86746475601b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:18.998936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d256b309-9bf8-44db-b02c-86746475601b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:19.007253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:19.019561 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1088/2185] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:19.020688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87aea2b-2628-4e81-b0b2-129771fdd26f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:19.021980 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c87aea2b-2628-4e81-b0b2-129771fdd26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:19.034116 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87aea2b-2628-4e81-b0b2-129771fdd26f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:19.040098 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a660bf-3ff2-4a8c-90fa-39c2278a70e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:19.040899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a660bf-3ff2-4a8c-90fa-39c2278a70e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:19.041195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a660bf-3ff2-4a8c-90fa-39c2278a70e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:19.041472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a660bf-3ff2-4a8c-90fa-39c2278a70e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:19.042196 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a660bf-3ff2-4a8c-90fa-39c2278a70e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:19.043980 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a660bf-3ff2-4a8c-90fa-39c2278a70e5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:19.054364 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1089/2186] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:19.055195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a660bf-3ff2-4a8c-90fa-39c2278a70e5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:19.065178 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a660bf-3ff2-4a8c-90fa-39c2278a70e5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:19.068910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3c128-5cef-4942-8ce8-995d937fc922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:19.069538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3c128-5cef-4942-8ce8-995d937fc922 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:19.069825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3c128-5cef-4942-8ce8-995d937fc922 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:19.070105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3c128-5cef-4942-8ce8-995d937fc922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:19.071732 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256b309-9bf8-44db-b02c-86746475601b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:19.073172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256b309-9bf8-44db-b02c-86746475601b service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:19.079267 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3c128-5cef-4942-8ce8-995d937fc922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:19.081344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3c128-5cef-4942-8ce8-995d937fc922 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:19.083949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1098/2187] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:19.084559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256b309-9bf8-44db-b02c-86746475601b service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:19.093920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256b309-9bf8-44db-b02c-86746475601b service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:19.096865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1090/2188] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:19.099739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558d8ee-724c-4bc5-ae72-614caf55df23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:19.100798 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558d8ee-724c-4bc5-ae72-614caf55df23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:19.101085 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558d8ee-724c-4bc5-ae72-614caf55df23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:19.101357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558d8ee-724c-4bc5-ae72-614caf55df23 admin admin] PATH_INFO: `/v3/projects/8576fbbfaba749afa20fe3a4305a6aa0/users/08e3a196d2074751823d4bb01c17c890/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:19.120544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558d8ee-724c-4bc5-ae72-614caf55df23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8576fbbfaba749afa20fe3a4305a6aa0, user_id=08e3a196d2074751823d4bb01c17c890, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:19.126148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558d8ee-724c-4bc5-ae72-614caf55df23 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:19.319571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1099/2189] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:19 2022] PUT /identity/v3/projects/8576fbbfaba749afa20fe3a4305a6aa0/users/08e3a196d2074751823d4bb01c17c890/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:19.330581 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c02db-7f66-4590-aeaf-a92250732484 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:19.330887 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c02db-7f66-4590-aeaf-a92250732484 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:19.331148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c02db-7f66-4590-aeaf-a92250732484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:19.367423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-867c02db-7f66-4590-aeaf-a92250732484 None None] MFA Rules not processed for user `08e3a196d2074751823d4bb01c17c890`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:19.367423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-867c02db-7f66-4590-aeaf-a92250732484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:19.484105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1091/2190] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:19.484640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558d8ee-724c-4bc5-ae72-614caf55df23 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:19.595045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558d8ee-724c-4bc5-ae72-614caf55df23 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:19.603394 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279464b-a1e5-41f8-9522-3bc27f470b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:19.604328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279464b-a1e5-41f8-9522-3bc27f470b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:19.611862 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279464b-a1e5-41f8-9522-3bc27f470b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:19.612239 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279464b-a1e5-41f8-9522-3bc27f470b31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:19.613803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279464b-a1e5-41f8-9522-3bc27f470b31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:19.616474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279464b-a1e5-41f8-9522-3bc27f470b31 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:19.735516 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c02db-7f66-4590-aeaf-a92250732484 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:19.736831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1100/2191] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:19 2022] POST /identity/v3/projects => generated 391 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:19.737144 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-867c02db-7f66-4590-aeaf-a92250732484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:19.740751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279464b-a1e5-41f8-9522-3bc27f470b31 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:19.761565 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c02db-7f66-4590-aeaf-a92250732484 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:19.762812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:19.832450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279464b-a1e5-41f8-9522-3bc27f470b31 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:19.833961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110dff5-bc75-40d8-b14a-4306fc9131e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:19.834837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110dff5-bc75-40d8-b14a-4306fc9131e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:19.835128 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110dff5-bc75-40d8-b14a-4306fc9131e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:19.835434 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110dff5-bc75-40d8-b14a-4306fc9131e2 admin admin] PATH_INFO: `/v3/projects/4afcf1169dca4aa7b0c2c4160c0bd743` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:19.836618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110dff5-bc75-40d8-b14a-4306fc9131e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4afcf1169dca4aa7b0c2c4160c0bd743)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:19.837101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29de7c-ac94-48b5-8b33-f9559575e25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:19.837771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29de7c-ac94-48b5-8b33-f9559575e25a admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:19.838058 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29de7c-ac94-48b5-8b33-f9559575e25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:19.838335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29de7c-ac94-48b5-8b33-f9559575e25a admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:19.838519 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110dff5-bc75-40d8-b14a-4306fc9131e2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:19.839026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29de7c-ac94-48b5-8b33-f9559575e25a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:19.841027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29de7c-ac94-48b5-8b33-f9559575e25a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:19.953471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1101/2192] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:19 2022] POST /identity/v3/users => generated 302 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:19.954538 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29de7c-ac94-48b5-8b33-f9559575e25a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:19.956260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9c29de7c-ac94-48b5-8b33-f9559575e25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:19.969393 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29de7c-ac94-48b5-8b33-f9559575e25a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:19.969568 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c110dff5-bc75-40d8-b14a-4306fc9131e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4afcf1169dca4aa7b0c2c4160c0bd743', 'request_id': 'req-c110dff5-bc75-40d8-b14a-4306fc9131e2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:19.976496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c110dff5-bc75-40d8-b14a-4306fc9131e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4afcf1169dca4aa7b0c2c4160c0bd743', 'request_id': 'req-c110dff5-bc75-40d8-b14a-4306fc9131e2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.025761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b5388-ea41-4671-95db-b6fc2dcf0d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:20.026554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b5388-ea41-4671-95db-b6fc2dcf0d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:20.026841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b5388-ea41-4671-95db-b6fc2dcf0d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:20.027124 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b5388-ea41-4671-95db-b6fc2dcf0d7a admin admin] PATH_INFO: `/v3/projects/f1046fd13d5343c491f716f6f5a71c93` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:20.028350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b5388-ea41-4671-95db-b6fc2dcf0d7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1046fd13d5343c491f716f6f5a71c93)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:20.030526 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b5388-ea41-4671-95db-b6fc2dcf0d7a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:20.070666 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c110dff5-bc75-40d8-b14a-4306fc9131e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4afcf1169dca4aa7b0c2c4160c0bd743', 'request_id': 'req-c110dff5-bc75-40d8-b14a-4306fc9131e2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.076268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:20.083204 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c110dff5-bc75-40d8-b14a-4306fc9131e2 admin admin] The token cache is being invalidate because project 4afcf1169dca4aa7b0c2c4160c0bd743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:20.083492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c110dff5-bc75-40d8-b14a-4306fc9131e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.084120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c110dff5-bc75-40d8-b14a-4306fc9131e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.085782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1092/2193] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:19 2022] DELETE /identity/v3/projects/4afcf1169dca4aa7b0c2c4160c0bd743 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:20.086640 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110dff5-bc75-40d8-b14a-4306fc9131e2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:20.087909 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c110dff5-bc75-40d8-b14a-4306fc9131e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:20.102165 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110dff5-bc75-40d8-b14a-4306fc9131e2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:20.111316 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe456348-53c1-495c-9b52-08bde9e5e39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:20.111316 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe456348-53c1-495c-9b52-08bde9e5e39e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:20.111316 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe456348-53c1-495c-9b52-08bde9e5e39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:20.111316 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe456348-53c1-495c-9b52-08bde9e5e39e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:20.111680 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe456348-53c1-495c-9b52-08bde9e5e39e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:20.111893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe456348-53c1-495c-9b52-08bde9e5e39e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:20.122189 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1093/2194] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:20.122974 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe456348-53c1-495c-9b52-08bde9e5e39e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:20.124653 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fe456348-53c1-495c-9b52-08bde9e5e39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:20.137311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe456348-53c1-495c-9b52-08bde9e5e39e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:20.142506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6292d64-9fd5-455f-bd26-e890afbf32c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:20.143631 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292d64-9fd5-455f-bd26-e890afbf32c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:20.143913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292d64-9fd5-455f-bd26-e890afbf32c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:20.144190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292d64-9fd5-455f-bd26-e890afbf32c0 admin admin] PATH_INFO: `/v3/users/3c4a91082e7549938ddd755e1a1a75c4` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:20.145407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6292d64-9fd5-455f-bd26-e890afbf32c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c4a91082e7549938ddd755e1a1a75c4)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:20.147360 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6292d64-9fd5-455f-bd26-e890afbf32c0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:20.179102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580b5388-ea41-4671-95db-b6fc2dcf0d7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1046fd13d5343c491f716f6f5a71c93', 'request_id': 'req-580b5388-ea41-4671-95db-b6fc2dcf0d7a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.185101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580b5388-ea41-4671-95db-b6fc2dcf0d7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1046fd13d5343c491f716f6f5a71c93', 'request_id': 'req-580b5388-ea41-4671-95db-b6fc2dcf0d7a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.300903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580b5388-ea41-4671-95db-b6fc2dcf0d7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1046fd13d5343c491f716f6f5a71c93', 'request_id': 'req-580b5388-ea41-4671-95db-b6fc2dcf0d7a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.312664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580b5388-ea41-4671-95db-b6fc2dcf0d7a admin admin] The token cache is being invalidate because project f1046fd13d5343c491f716f6f5a71c93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:20.312953 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580b5388-ea41-4671-95db-b6fc2dcf0d7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.313623 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580b5388-ea41-4671-95db-b6fc2dcf0d7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.315433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1102/2195] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:19 2022] DELETE /identity/v3/projects/f1046fd13d5343c491f716f6f5a71c93 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:20.316176 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b5388-ea41-4671-95db-b6fc2dcf0d7a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:20.317460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-580b5388-ea41-4671-95db-b6fc2dcf0d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:20.371053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d6292d64-9fd5-455f-bd26-e890afbf32c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c4a91082e7549938ddd755e1a1a75c4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.394934 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b5388-ea41-4671-95db-b6fc2dcf0d7a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:20.397052 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d6292d64-9fd5-455f-bd26-e890afbf32c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c4a91082e7549938ddd755e1a1a75c4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.402277 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3cb8d-723c-4ba2-ac0f-8282c956bb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:20.403606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3cb8d-723c-4ba2-ac0f-8282c956bb0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:20.404152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3cb8d-723c-4ba2-ac0f-8282c956bb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:20.404834 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3cb8d-723c-4ba2-ac0f-8282c956bb0c admin admin] PATH_INFO: `/v3/projects/1047d27e59074281b2dd61dbfe910beb/users/dc6274e0ee844f09a9e5a730ed0f6ead/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:20.408400 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d6292d64-9fd5-455f-bd26-e890afbf32c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c4a91082e7549938ddd755e1a1a75c4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.420499 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d6292d64-9fd5-455f-bd26-e890afbf32c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c4a91082e7549938ddd755e1a1a75c4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.421215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d6292d64-9fd5-455f-bd26-e890afbf32c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c4a91082e7549938ddd755e1a1a75c4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.425140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3cb8d-723c-4ba2-ac0f-8282c956bb0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1047d27e59074281b2dd61dbfe910beb, user_id=dc6274e0ee844f09a9e5a730ed0f6ead, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:20.428419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3cb8d-723c-4ba2-ac0f-8282c956bb0c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:20.498161 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1094/2196] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:20 2022] DELETE /identity/v3/users/3c4a91082e7549938ddd755e1a1a75c4 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:20.498397 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1103/2197] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:20 2022] PUT /identity/v3/projects/1047d27e59074281b2dd61dbfe910beb/users/dc6274e0ee844f09a9e5a730ed0f6ead/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:20.498496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6292d64-9fd5-455f-bd26-e890afbf32c0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:20.498496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d6292d64-9fd5-455f-bd26-e890afbf32c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:20.500547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3cb8d-723c-4ba2-ac0f-8282c956bb0c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:20.503233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e8f3cb8d-723c-4ba2-ac0f-8282c956bb0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:20.556367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:20.589777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3cb8d-723c-4ba2-ac0f-8282c956bb0c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:20.617332 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6292d64-9fd5-455f-bd26-e890afbf32c0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:20.622943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615b7fd-4a1d-44e9-8770-6acaef4d7fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:20.623868 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615b7fd-4a1d-44e9-8770-6acaef4d7fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:20.624176 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615b7fd-4a1d-44e9-8770-6acaef4d7fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:20.624450 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615b7fd-4a1d-44e9-8770-6acaef4d7fb0 admin admin] PATH_INFO: `/v3/users/513d0e20e17e4ffb9a393b72cfea67f0` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:20.625698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615b7fd-4a1d-44e9-8770-6acaef4d7fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=513d0e20e17e4ffb9a393b72cfea67f0)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:20.628304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615b7fd-4a1d-44e9-8770-6acaef4d7fb0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:20.685628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec7007-2e2b-454a-84bd-b79b7a3b665a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:20.686320 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec7007-2e2b-454a-84bd-b79b7a3b665a admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:20.686609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec7007-2e2b-454a-84bd-b79b7a3b665a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:20.686887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec7007-2e2b-454a-84bd-b79b7a3b665a admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:20.687897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec7007-2e2b-454a-84bd-b79b7a3b665a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:20.689945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec7007-2e2b-454a-84bd-b79b7a3b665a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:20.700308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1104/2198] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:20.701749 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef92a6-ea98-4d9e-a226-14b72b5019d6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:20.702049 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef92a6-ea98-4d9e-a226-14b72b5019d6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:20.702336 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef92a6-ea98-4d9e-a226-14b72b5019d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:20.721182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6aef92a6-ea98-4d9e-a226-14b72b5019d6 None None] MFA Rules not processed for user `c95b2f1348424e6fa05c9599d4e3611a`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:20.722672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6aef92a6-ea98-4d9e-a226-14b72b5019d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:20.792443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1105/2199] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:20.792443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef92a6-ea98-4d9e-a226-14b72b5019d6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:20.814316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef92a6-ea98-4d9e-a226-14b72b5019d6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:20.816149 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3787aa-6e1b-4714-bb3c-4e59213c7e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:20.816557 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3787aa-6e1b-4714-bb3c-4e59213c7e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:20.816843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3787aa-6e1b-4714-bb3c-4e59213c7e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:20.817123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3787aa-6e1b-4714-bb3c-4e59213c7e61 admin admin] PATH_INFO: `/v3/projects/1047d27e59074281b2dd61dbfe910beb/users/dc6274e0ee844f09a9e5a730ed0f6ead/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:20.819535 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3787aa-6e1b-4714-bb3c-4e59213c7e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1047d27e59074281b2dd61dbfe910beb, user_id=dc6274e0ee844f09a9e5a730ed0f6ead, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:20.822208 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3787aa-6e1b-4714-bb3c-4e59213c7e61 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:20.835662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1106/2200] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:20 2022] PUT /identity/v3/projects/1047d27e59074281b2dd61dbfe910beb/users/dc6274e0ee844f09a9e5a730ed0f6ead/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:20.847814 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff04ea-e552-43fc-8ea7-caf226ff74c1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:20.847983 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff04ea-e552-43fc-8ea7-caf226ff74c1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:20.848203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff04ea-e552-43fc-8ea7-caf226ff74c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:20.884006 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2615b7fd-4a1d-44e9-8770-6acaef4d7fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '513d0e20e17e4ffb9a393b72cfea67f0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.908354 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-daff04ea-e552-43fc-8ea7-caf226ff74c1 None None] MFA Rules not processed for user `dc6274e0ee844f09a9e5a730ed0f6ead`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:20.911245 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-daff04ea-e552-43fc-8ea7-caf226ff74c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:20.912788 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2615b7fd-4a1d-44e9-8770-6acaef4d7fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '513d0e20e17e4ffb9a393b72cfea67f0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.915094 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2615b7fd-4a1d-44e9-8770-6acaef4d7fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '513d0e20e17e4ffb9a393b72cfea67f0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.927110 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2615b7fd-4a1d-44e9-8770-6acaef4d7fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '513d0e20e17e4ffb9a393b72cfea67f0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:20.928122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2615b7fd-4a1d-44e9-8770-6acaef4d7fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '513d0e20e17e4ffb9a393b72cfea67f0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:21.026140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1107/2201] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:21.035644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1095/2202] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:20 2022] DELETE /identity/v3/users/513d0e20e17e4ffb9a393b72cfea67f0 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:21.045190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff04ea-e552-43fc-8ea7-caf226ff74c1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:21.047463 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-daff04ea-e552-43fc-8ea7-caf226ff74c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:21.150290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:21.159377 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff04ea-e552-43fc-8ea7-caf226ff74c1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:21.211558 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692f185-b427-43c3-bcfb-0d9169f4b610 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:21.212212 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692f185-b427-43c3-bcfb-0d9169f4b610 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:21.212547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692f185-b427-43c3-bcfb-0d9169f4b610 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:21.212828 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692f185-b427-43c3-bcfb-0d9169f4b610 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:21.284559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615b7fd-4a1d-44e9-8770-6acaef4d7fb0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:21.286375 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2615b7fd-4a1d-44e9-8770-6acaef4d7fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:21.294193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692f185-b427-43c3-bcfb-0d9169f4b610 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:21.295819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692f185-b427-43c3-bcfb-0d9169f4b610 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:21.324894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1108/2203] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:21.399626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692f185-b427-43c3-bcfb-0d9169f4b610 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:21.409881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692f185-b427-43c3-bcfb-0d9169f4b610 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:21.413806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3db494-755f-4114-be89-e697eca4b9d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:21.414522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3db494-755f-4114-be89-e697eca4b9d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:21.414864 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3db494-755f-4114-be89-e697eca4b9d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:21.415221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3db494-755f-4114-be89-e697eca4b9d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:21.417274 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fa3db494-755f-4114-be89-e697eca4b9d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:21.428714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615b7fd-4a1d-44e9-8770-6acaef4d7fb0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:21.434672 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316afbd-fcb6-4fa5-a887-fddd620aad1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:21.435754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316afbd-fcb6-4fa5-a887-fddd620aad1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:21.435889 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316afbd-fcb6-4fa5-a887-fddd620aad1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:21.436174 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316afbd-fcb6-4fa5-a887-fddd620aad1b admin admin] PATH_INFO: `/v3/projects/db8d818144084415bb59bdcc6cb53712` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:21.437852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316afbd-fcb6-4fa5-a887-fddd620aad1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=db8d818144084415bb59bdcc6cb53712)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:21.440513 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316afbd-fcb6-4fa5-a887-fddd620aad1b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:21.497601 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3db494-755f-4114-be89-e697eca4b9d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:21.499117 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3db494-755f-4114-be89-e697eca4b9d7 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:21.516111 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1109/2204] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:20:21.556505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a316afbd-fcb6-4fa5-a887-fddd620aad1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db8d818144084415bb59bdcc6cb53712', 'request_id': 'req-a316afbd-fcb6-4fa5-a887-fddd620aad1b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:21.566723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a316afbd-fcb6-4fa5-a887-fddd620aad1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db8d818144084415bb59bdcc6cb53712', 'request_id': 'req-a316afbd-fcb6-4fa5-a887-fddd620aad1b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:21.568235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3db494-755f-4114-be89-e697eca4b9d7 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:21.570675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:21.573893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fa3db494-755f-4114-be89-e697eca4b9d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:21.622065 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3db494-755f-4114-be89-e697eca4b9d7 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:21.659548 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a316afbd-fcb6-4fa5-a887-fddd620aad1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db8d818144084415bb59bdcc6cb53712', 'request_id': 'req-a316afbd-fcb6-4fa5-a887-fddd620aad1b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:21.687911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a316afbd-fcb6-4fa5-a887-fddd620aad1b admin admin] The token cache is being invalidate because project db8d818144084415bb59bdcc6cb53712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:21.687911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a316afbd-fcb6-4fa5-a887-fddd620aad1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:21.687911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a316afbd-fcb6-4fa5-a887-fddd620aad1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:21.687911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1096/2205] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:21 2022] DELETE /identity/v3/projects/db8d818144084415bb59bdcc6cb53712 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:21.687911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316afbd-fcb6-4fa5-a887-fddd620aad1b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:21.688236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a316afbd-fcb6-4fa5-a887-fddd620aad1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:21.730188 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761cd36-1449-424e-811d-961359c3720c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:21.731096 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761cd36-1449-424e-811d-961359c3720c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:21.731403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761cd36-1449-424e-811d-961359c3720c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:21.731677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761cd36-1449-424e-811d-961359c3720c admin admin] PATH_INFO: `/v3/users/46aa9e9140014bc3b555de7f1e2b1df4` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:21.733000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1761cd36-1449-424e-811d-961359c3720c admin admin] RBAC: Authorizing `identity:delete_user(user_id=46aa9e9140014bc3b555de7f1e2b1df4)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:21.735023 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1761cd36-1449-424e-811d-961359c3720c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:21.764291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316afbd-fcb6-4fa5-a887-fddd620aad1b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:21.773117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:21.776053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:21.776341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:21.776672 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f admin admin] PATH_INFO: `/v3/projects/670c34bbd1bd40979e2e3c331a1790ae` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:21.778525 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f admin admin] RBAC: Authorizing `identity:delete_project(project_id=670c34bbd1bd40979e2e3c331a1790ae)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:21.785147 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:21.957573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1761cd36-1449-424e-811d-961359c3720c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46aa9e9140014bc3b555de7f1e2b1df4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:21.958534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670c34bbd1bd40979e2e3c331a1790ae', 'request_id': 'req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:21.966333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670c34bbd1bd40979e2e3c331a1790ae', 'request_id': 'req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:21.971271 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1761cd36-1449-424e-811d-961359c3720c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46aa9e9140014bc3b555de7f1e2b1df4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:22.060898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670c34bbd1bd40979e2e3c331a1790ae', 'request_id': 'req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:22.075666 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f admin admin] The token cache is being invalidate because project 670c34bbd1bd40979e2e3c331a1790ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:22.075666 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:22.075666 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:22.081476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1097/2206] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:21 2022] DELETE /identity/v3/projects/670c34bbd1bd40979e2e3c331a1790ae => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:22.081476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:22.081476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:22.108261 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:22.155234 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb8ff6-3bfa-411e-9e76-254e17c2870f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:22.160491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea554967-545f-453a-9b78-03065e803e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:22.161361 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea554967-545f-453a-9b78-03065e803e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:22.161644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea554967-545f-453a-9b78-03065e803e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:22.161917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea554967-545f-453a-9b78-03065e803e59 admin admin] PATH_INFO: `/v3/users/bcb3b268867840e7b8e1390379291f43` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:22.163092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea554967-545f-453a-9b78-03065e803e59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb3b268867840e7b8e1390379291f43)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:22.165042 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea554967-545f-453a-9b78-03065e803e59 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:22.197396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1761cd36-1449-424e-811d-961359c3720c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46aa9e9140014bc3b555de7f1e2b1df4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:22.198058 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1761cd36-1449-424e-811d-961359c3720c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46aa9e9140014bc3b555de7f1e2b1df4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:22.205573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1761cd36-1449-424e-811d-961359c3720c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46aa9e9140014bc3b555de7f1e2b1df4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:22.211530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1110/2207] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:21 2022] DELETE /identity/v3/users/46aa9e9140014bc3b555de7f1e2b1df4 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:22.212793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2a279-e6de-498a-8243-69dc1bfc9500 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:22.213081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2a279-e6de-498a-8243-69dc1bfc9500 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:22.213361 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2a279-e6de-498a-8243-69dc1bfc9500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:22.231380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5bd2a279-e6de-498a-8243-69dc1bfc9500 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:22.232595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5bd2a279-e6de-498a-8243-69dc1bfc9500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:22.306970 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1111/2208] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:22.308461 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c102ba-4d6d-4705-b9f6-e386daec8b88 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:22.308882 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c102ba-4d6d-4705-b9f6-e386daec8b88 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:22.309285 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c102ba-4d6d-4705-b9f6-e386daec8b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:22.332426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b1c102ba-4d6d-4705-b9f6-e386daec8b88 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:22.333640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b1c102ba-4d6d-4705-b9f6-e386daec8b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:22.341458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1112/2209] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:22.342668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca408079-f10e-4b50-92c9-8f65dca24bf9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:22.342950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca408079-f10e-4b50-92c9-8f65dca24bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:22.343251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca408079-f10e-4b50-92c9-8f65dca24bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:22.359070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:22.364473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ca408079-f10e-4b50-92c9-8f65dca24bf9 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:22.365662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ca408079-f10e-4b50-92c9-8f65dca24bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:22.373641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1113/2210] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:22.374387 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca408079-f10e-4b50-92c9-8f65dca24bf9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:22.375583 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ca408079-f10e-4b50-92c9-8f65dca24bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:22.406934 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ea554967-545f-453a-9b78-03065e803e59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb3b268867840e7b8e1390379291f43'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:22.425610 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ea554967-545f-453a-9b78-03065e803e59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb3b268867840e7b8e1390379291f43'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:22.427076 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ea554967-545f-453a-9b78-03065e803e59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb3b268867840e7b8e1390379291f43'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:22.435630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca408079-f10e-4b50-92c9-8f65dca24bf9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:22.436846 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ea554967-545f-453a-9b78-03065e803e59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb3b268867840e7b8e1390379291f43'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:22.438324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ea554967-545f-453a-9b78-03065e803e59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb3b268867840e7b8e1390379291f43'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:22.439928 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e0fe6-3a42-4fe3-82ea-132d6dd3495c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:22.440599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e0fe6-3a42-4fe3-82ea-132d6dd3495c service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:22.440880 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e0fe6-3a42-4fe3-82ea-132d6dd3495c service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:22.441156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e0fe6-3a42-4fe3-82ea-132d6dd3495c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:22.443824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9e1e0fe6-3a42-4fe3-82ea-132d6dd3495c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:22.458395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e0fe6-3a42-4fe3-82ea-132d6dd3495c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:22.459891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e0fe6-3a42-4fe3-82ea-132d6dd3495c service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:22.471339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1114/2211] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:22.472031 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e0fe6-3a42-4fe3-82ea-132d6dd3495c service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:22.475032 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9e1e0fe6-3a42-4fe3-82ea-132d6dd3495c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:22.486492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e0fe6-3a42-4fe3-82ea-132d6dd3495c service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:22.490532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e31cd-bd26-4a8c-96cb-7758e67408f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:22.491555 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e31cd-bd26-4a8c-96cb-7758e67408f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:22.491710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e31cd-bd26-4a8c-96cb-7758e67408f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:22.492125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e31cd-bd26-4a8c-96cb-7758e67408f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:22.493145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e31cd-bd26-4a8c-96cb-7758e67408f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:22.494038 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e31cd-bd26-4a8c-96cb-7758e67408f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:22.495296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e31cd-bd26-4a8c-96cb-7758e67408f7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:22.507938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1115/2212] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:20:22.512204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e31cd-bd26-4a8c-96cb-7758e67408f7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:22.522810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e31cd-bd26-4a8c-96cb-7758e67408f7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:22.528344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bacd439-813a-4309-a62b-aaa1d413295a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:22.528596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1098/2213] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:22 2022] DELETE /identity/v3/users/bcb3b268867840e7b8e1390379291f43 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:22.532312 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacd439-813a-4309-a62b-aaa1d413295a admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:22.533237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacd439-813a-4309-a62b-aaa1d413295a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:22.533527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacd439-813a-4309-a62b-aaa1d413295a admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:22.534283 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bacd439-813a-4309-a62b-aaa1d413295a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:22.536376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bacd439-813a-4309-a62b-aaa1d413295a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:22.650130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1116/2214] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:22 2022] POST /identity/v3/projects => generated 399 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:22.654641 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea554967-545f-453a-9b78-03065e803e59 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:22.747222 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea554967-545f-453a-9b78-03065e803e59 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:22.826822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:22.847169 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25d73b-5cd5-4a81-9456-d7eb577f05e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:22.849781 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25d73b-5cd5-4a81-9456-d7eb577f05e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:22.850296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25d73b-5cd5-4a81-9456-d7eb577f05e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:22.851370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25d73b-5cd5-4a81-9456-d7eb577f05e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:22.852281 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25d73b-5cd5-4a81-9456-d7eb577f05e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:22.856786 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25d73b-5cd5-4a81-9456-d7eb577f05e0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:23.017925 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:23.025653 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1099/2215] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:22 2022] POST /identity/v3/users => generated 300 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:23.025954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bacd439-813a-4309-a62b-aaa1d413295a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:23.028572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0bacd439-813a-4309-a62b-aaa1d413295a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:23.034045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25d73b-5cd5-4a81-9456-d7eb577f05e0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:23.051244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bacd439-813a-4309-a62b-aaa1d413295a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:23.054370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25d73b-5cd5-4a81-9456-d7eb577f05e0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:23.066143 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:23.067047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:23.072190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:23.072190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286 admin admin] PATH_INFO: `/v3/projects/b359beefcd8943a8a7761abc331e6c5a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:23.072190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b359beefcd8943a8a7761abc331e6c5a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:23.072745 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed73a7-b3a9-48d0-b504-c9ff313a3135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:23.072745 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed73a7-b3a9-48d0-b504-c9ff313a3135 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:23.072937 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed73a7-b3a9-48d0-b504-c9ff313a3135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:23.073199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed73a7-b3a9-48d0-b504-c9ff313a3135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:23.074118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed73a7-b3a9-48d0-b504-c9ff313a3135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:23.075765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed73a7-b3a9-48d0-b504-c9ff313a3135 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:23.078275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:23.093576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1100/2216] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:23.095642 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed73a7-b3a9-48d0-b504-c9ff313a3135 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:23.098712 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c5ed73a7-b3a9-48d0-b504-c9ff313a3135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:23.119961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed73a7-b3a9-48d0-b504-c9ff313a3135 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:23.125695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd6438-97cd-4bd6-b358-b432c63e0e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:23.126896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd6438-97cd-4bd6-b358-b432c63e0e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:23.127306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd6438-97cd-4bd6-b358-b432c63e0e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:23.127587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd6438-97cd-4bd6-b358-b432c63e0e24 admin admin] PATH_INFO: `/v3/projects/77496a24e01e461986a18995f456c10c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:23.129437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd6438-97cd-4bd6-b358-b432c63e0e24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77496a24e01e461986a18995f456c10c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:23.131479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd6438-97cd-4bd6-b358-b432c63e0e24 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:23.232638 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b359beefcd8943a8a7761abc331e6c5a', 'request_id': 'req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:23.238958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b359beefcd8943a8a7761abc331e6c5a', 'request_id': 'req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:23.292355 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b6dd6438-97cd-4bd6-b358-b432c63e0e24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77496a24e01e461986a18995f456c10c', 'request_id': 'req-b6dd6438-97cd-4bd6-b358-b432c63e0e24'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:23.299085 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b6dd6438-97cd-4bd6-b358-b432c63e0e24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77496a24e01e461986a18995f456c10c', 'request_id': 'req-b6dd6438-97cd-4bd6-b358-b432c63e0e24'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:23.310927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b359beefcd8943a8a7761abc331e6c5a', 'request_id': 'req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:23.322632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286 admin admin] The token cache is being invalidate because project b359beefcd8943a8a7761abc331e6c5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:23.322943 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:23.323601 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:23.325168 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1117/2217] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:23 2022] DELETE /identity/v3/projects/b359beefcd8943a8a7761abc331e6c5a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:23.325924 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:23.327221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:23.405537 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b6dd6438-97cd-4bd6-b358-b432c63e0e24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77496a24e01e461986a18995f456c10c', 'request_id': 'req-b6dd6438-97cd-4bd6-b358-b432c63e0e24'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:23.436951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b6dd6438-97cd-4bd6-b358-b432c63e0e24 admin admin] The token cache is being invalidate because project 77496a24e01e461986a18995f456c10c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:23.445688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b6dd6438-97cd-4bd6-b358-b432c63e0e24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:23.445688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b6dd6438-97cd-4bd6-b358-b432c63e0e24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:23.445688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1101/2218] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:23 2022] DELETE /identity/v3/projects/77496a24e01e461986a18995f456c10c => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:23.448810 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555480cc-116d-4dca-8322-57fc70bb2f18 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:23.449715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555480cc-116d-4dca-8322-57fc70bb2f18 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:23.450142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555480cc-116d-4dca-8322-57fc70bb2f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:23.491318 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa1f1f-eefc-4db7-bfb5-a903cbbbf286 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:23.494959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-555480cc-116d-4dca-8322-57fc70bb2f18 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:23.501987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-555480cc-116d-4dca-8322-57fc70bb2f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:23.595172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0612e7c-13ea-4c9d-baa8-b0f64cdf7063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:23.595978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0612e7c-13ea-4c9d-baa8-b0f64cdf7063 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:23.596673 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0612e7c-13ea-4c9d-baa8-b0f64cdf7063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:23.596960 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0612e7c-13ea-4c9d-baa8-b0f64cdf7063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:23.597886 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0612e7c-13ea-4c9d-baa8-b0f64cdf7063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:23.599387 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0612e7c-13ea-4c9d-baa8-b0f64cdf7063 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:23.612369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1102/2219] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:23.612369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555480cc-116d-4dca-8322-57fc70bb2f18 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:23.612369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-555480cc-116d-4dca-8322-57fc70bb2f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:23.615260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1118/2220] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:23.615260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b072e72-d2a3-4319-b0e5-41b3519b7ac4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:23.615260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b072e72-d2a3-4319-b0e5-41b3519b7ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:23.615260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b072e72-d2a3-4319-b0e5-41b3519b7ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:23.623896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555480cc-116d-4dca-8322-57fc70bb2f18 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:23.630858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b075299-f011-4a11-8d5e-95c03c777ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:23.632253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b075299-f011-4a11-8d5e-95c03c777ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:23.632623 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b075299-f011-4a11-8d5e-95c03c777ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:23.632964 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b075299-f011-4a11-8d5e-95c03c777ea4 admin admin] PATH_INFO: `/v3/users/5143a5d2d2e84d8396174ae3dc505c54` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:23.634265 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b075299-f011-4a11-8d5e-95c03c777ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5143a5d2d2e84d8396174ae3dc505c54)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:23.636470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b075299-f011-4a11-8d5e-95c03c777ea4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:23.652278 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9b072e72-d2a3-4319-b0e5-41b3519b7ac4 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:23.653471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9b072e72-d2a3-4319-b0e5-41b3519b7ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:23.662914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1119/2221] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:23.666568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b072e72-d2a3-4319-b0e5-41b3519b7ac4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:23.667920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9b072e72-d2a3-4319-b0e5-41b3519b7ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:23.682516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b072e72-d2a3-4319-b0e5-41b3519b7ac4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:23.687320 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8cf790-cc12-461d-b333-32203010b903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:23.688539 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cf790-cc12-461d-b333-32203010b903 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:23.689876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cf790-cc12-461d-b333-32203010b903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:23.690165 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cf790-cc12-461d-b333-32203010b903 admin admin] PATH_INFO: `/v3/projects/3466295427d947bb901d3e41e25c0581/users/f3fea01118f54873a5f59d6c22b74205/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:23.709132 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8cf790-cc12-461d-b333-32203010b903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3466295427d947bb901d3e41e25c0581, user_id=f3fea01118f54873a5f59d6c22b74205, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:23.712284 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8cf790-cc12-461d-b333-32203010b903 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:23.833546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1120/2222] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:23 2022] PUT /identity/v3/projects/3466295427d947bb901d3e41e25c0581/users/f3fea01118f54873a5f59d6c22b74205/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:23.834538 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8cf790-cc12-461d-b333-32203010b903 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:23.862015 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b075299-f011-4a11-8d5e-95c03c777ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5143a5d2d2e84d8396174ae3dc505c54'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:23.874329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b075299-f011-4a11-8d5e-95c03c777ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5143a5d2d2e84d8396174ae3dc505c54'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:23.875043 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b075299-f011-4a11-8d5e-95c03c777ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5143a5d2d2e84d8396174ae3dc505c54'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:23.885910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b075299-f011-4a11-8d5e-95c03c777ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5143a5d2d2e84d8396174ae3dc505c54'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:23.886943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b075299-f011-4a11-8d5e-95c03c777ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5143a5d2d2e84d8396174ae3dc505c54'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:23.910359 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8cf790-cc12-461d-b333-32203010b903 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:23.915986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad891d-3aa6-404d-b491-79d1e10745e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:23.917043 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad891d-3aa6-404d-b491-79d1e10745e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:23.917912 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad891d-3aa6-404d-b491-79d1e10745e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:23.917912 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad891d-3aa6-404d-b491-79d1e10745e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:23.918328 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad891d-3aa6-404d-b491-79d1e10745e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:23.918640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad891d-3aa6-404d-b491-79d1e10745e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:23.920355 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad891d-3aa6-404d-b491-79d1e10745e0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:23.933542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1121/2223] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:20:23.935139 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe5356-4d85-4c26-a146-a73e6e6ed2be None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:23.935654 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe5356-4d85-4c26-a146-a73e6e6ed2be None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:23.936178 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe5356-4d85-4c26-a146-a73e6e6ed2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:23.965093 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-acfe5356-4d85-4c26-a146-a73e6e6ed2be None None] MFA Rules not processed for user `f3fea01118f54873a5f59d6c22b74205`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:23.966343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-acfe5356-4d85-4c26-a146-a73e6e6ed2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:23.997998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1103/2224] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:23 2022] DELETE /identity/v3/users/5143a5d2d2e84d8396174ae3dc505c54 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:24.002407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b075299-f011-4a11-8d5e-95c03c777ea4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:24.003769 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1b075299-f011-4a11-8d5e-95c03c777ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:24.076224 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1122/2225] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:24.078046 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25df6f-1383-41dc-9228-ed5b0843396f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:24.078526 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25df6f-1383-41dc-9228-ed5b0843396f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:24.079013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25df6f-1383-41dc-9228-ed5b0843396f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:24.101400 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6c25df6f-1383-41dc-9228-ed5b0843396f None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:24.103074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6c25df6f-1383-41dc-9228-ed5b0843396f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:24.122230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b075299-f011-4a11-8d5e-95c03c777ea4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:24.137737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd5be3-6107-45b7-869b-73cc4945638a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:24.139100 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd5be3-6107-45b7-869b-73cc4945638a admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:24.139648 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd5be3-6107-45b7-869b-73cc4945638a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:24.139935 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd5be3-6107-45b7-869b-73cc4945638a admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:24.140692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd5be3-6107-45b7-869b-73cc4945638a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:24.142910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd5be3-6107-45b7-869b-73cc4945638a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:24.197530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1123/2226] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:24.198631 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25df6f-1383-41dc-9228-ed5b0843396f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:24.204033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6c25df6f-1383-41dc-9228-ed5b0843396f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:24.216568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25df6f-1383-41dc-9228-ed5b0843396f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:24.222035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8965ef6-d567-4ffe-8827-08b47924e1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:24.222706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8965ef6-d567-4ffe-8827-08b47924e1d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:24.223013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8965ef6-d567-4ffe-8827-08b47924e1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:24.223327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8965ef6-d567-4ffe-8827-08b47924e1d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:24.224030 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8965ef6-d567-4ffe-8827-08b47924e1d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:24.225936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8965ef6-d567-4ffe-8827-08b47924e1d1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:24.284680 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:24.295001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1104/2227] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:23 2022] POST /identity/v3/projects => generated 391 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:24.296168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd5be3-6107-45b7-869b-73cc4945638a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:24.298061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d2cd5be3-6107-45b7-869b-73cc4945638a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:24.370013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd5be3-6107-45b7-869b-73cc4945638a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:24.374080 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813235f-ec45-4d1a-8bb3-2ff2dac6f11d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:24.376682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813235f-ec45-4d1a-8bb3-2ff2dac6f11d service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:24.377162 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813235f-ec45-4d1a-8bb3-2ff2dac6f11d service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:24.377551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813235f-ec45-4d1a-8bb3-2ff2dac6f11d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:24.389458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1124/2228] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:24 2022] POST /identity/v3/projects => generated 401 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:24.392872 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8965ef6-d567-4ffe-8827-08b47924e1d1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:24.455732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813235f-ec45-4d1a-8bb3-2ff2dac6f11d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:24.457406 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813235f-ec45-4d1a-8bb3-2ff2dac6f11d service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:24.463381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8965ef6-d567-4ffe-8827-08b47924e1d1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:24.469551 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751b9d5-0768-498f-adb4-21e20df8382c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:24.470333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1105/2229] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:24.471315 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813235f-ec45-4d1a-8bb3-2ff2dac6f11d service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:24.471629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751b9d5-0768-498f-adb4-21e20df8382c admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:24.472438 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751b9d5-0768-498f-adb4-21e20df8382c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:24.473513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751b9d5-0768-498f-adb4-21e20df8382c admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:24.476149 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751b9d5-0768-498f-adb4-21e20df8382c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:24.478288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751b9d5-0768-498f-adb4-21e20df8382c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:24.483657 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813235f-ec45-4d1a-8bb3-2ff2dac6f11d service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:24.492053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff977e68-d47b-4d24-b5c3-abb578046425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:24.492842 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff977e68-d47b-4d24-b5c3-abb578046425 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:24.493221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff977e68-d47b-4d24-b5c3-abb578046425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:24.493635 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff977e68-d47b-4d24-b5c3-abb578046425 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:24.494426 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff977e68-d47b-4d24-b5c3-abb578046425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:24.497379 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff977e68-d47b-4d24-b5c3-abb578046425 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:24.528797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:24.566232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1125/2230] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:24 2022] POST /identity/v3/users => generated 296 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:24.570385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751b9d5-0768-498f-adb4-21e20df8382c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:24.586442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751b9d5-0768-498f-adb4-21e20df8382c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:24.601601 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bd01f-ede1-4a17-9390-9054a859158b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:24.602359 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bd01f-ede1-4a17-9390-9054a859158b admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:24.602677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bd01f-ede1-4a17-9390-9054a859158b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:24.603078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bd01f-ede1-4a17-9390-9054a859158b admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:24.603952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bd01f-ede1-4a17-9390-9054a859158b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:24.606255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1106/2231] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:24 2022] POST /identity/v3/users => generated 307 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:24.606487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bd01f-ede1-4a17-9390-9054a859158b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:24.607564 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff977e68-d47b-4d24-b5c3-abb578046425 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:24.624402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1126/2232] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:24.626096 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bd01f-ede1-4a17-9390-9054a859158b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:24.631345 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff977e68-d47b-4d24-b5c3-abb578046425 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:24.638162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bd01f-ede1-4a17-9390-9054a859158b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:24.645547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91deecce-e24c-4f97-95c0-cc46c26cbe45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:24.645547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91deecce-e24c-4f97-95c0-cc46c26cbe45 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:24.645547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91deecce-e24c-4f97-95c0-cc46c26cbe45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:24.645547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91deecce-e24c-4f97-95c0-cc46c26cbe45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:24.646119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91deecce-e24c-4f97-95c0-cc46c26cbe45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:24.646119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91deecce-e24c-4f97-95c0-cc46c26cbe45 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:24.648228 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760941f9-1b26-4e28-987f-84b2e9a6b0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:24.648966 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760941f9-1b26-4e28-987f-84b2e9a6b0a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:24.649256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760941f9-1b26-4e28-987f-84b2e9a6b0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:24.649684 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760941f9-1b26-4e28-987f-84b2e9a6b0a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:24.650519 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760941f9-1b26-4e28-987f-84b2e9a6b0a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:24.652864 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760941f9-1b26-4e28-987f-84b2e9a6b0a1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:24.664856 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1107/2233] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:24.667483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91deecce-e24c-4f97-95c0-cc46c26cbe45 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:24.667615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1127/2234] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:24.670549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760941f9-1b26-4e28-987f-84b2e9a6b0a1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:24.680684 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91deecce-e24c-4f97-95c0-cc46c26cbe45 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:24.685395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760941f9-1b26-4e28-987f-84b2e9a6b0a1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:24.689932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462eb5b7-9184-4b33-893d-2a33d1a3854e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:24.691284 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462eb5b7-9184-4b33-893d-2a33d1a3854e admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:24.691792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462eb5b7-9184-4b33-893d-2a33d1a3854e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:24.692207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462eb5b7-9184-4b33-893d-2a33d1a3854e admin admin] PATH_INFO: `/v3/projects/a53d3dafe3c8454188b83fdf000b8a5d/users/51beaf0b777043d1864c84dde454fa57/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:24.693426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56978cc-a7a6-499f-99b7-9fdd13aae2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:24.694504 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56978cc-a7a6-499f-99b7-9fdd13aae2ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:24.695099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56978cc-a7a6-499f-99b7-9fdd13aae2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:24.695522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56978cc-a7a6-499f-99b7-9fdd13aae2ad admin admin] PATH_INFO: `/v3/projects/ad86395efbb949d89e70d8f13f47ecc3/users/d721c3d69b2c4865ae23de3110c3c909/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:24.715003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56978cc-a7a6-499f-99b7-9fdd13aae2ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad86395efbb949d89e70d8f13f47ecc3, user_id=d721c3d69b2c4865ae23de3110c3c909, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:24.720778 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56978cc-a7a6-499f-99b7-9fdd13aae2ad admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:24.722851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462eb5b7-9184-4b33-893d-2a33d1a3854e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a53d3dafe3c8454188b83fdf000b8a5d, user_id=51beaf0b777043d1864c84dde454fa57, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:24.726561 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462eb5b7-9184-4b33-893d-2a33d1a3854e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:24.794298 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1108/2235] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:24 2022] PUT /identity/v3/projects/a53d3dafe3c8454188b83fdf000b8a5d/users/51beaf0b777043d1864c84dde454fa57/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:24.798585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462eb5b7-9184-4b33-893d-2a33d1a3854e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:24.810952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1128/2236] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:24 2022] PUT /identity/v3/projects/ad86395efbb949d89e70d8f13f47ecc3/users/d721c3d69b2c4865ae23de3110c3c909/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:24.812534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba56f5-b423-4ca2-be15-56957df6cbfb None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:24.812822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba56f5-b423-4ca2-be15-56957df6cbfb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:24.818082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba56f5-b423-4ca2-be15-56957df6cbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:24.870654 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-16ba56f5-b423-4ca2-be15-56957df6cbfb None None] MFA Rules not processed for user `d721c3d69b2c4865ae23de3110c3c909`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:24.872185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-16ba56f5-b423-4ca2-be15-56957df6cbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:24.931638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462eb5b7-9184-4b33-893d-2a33d1a3854e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:24.976849 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1129/2237] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:24.988491 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba56f5-b423-4ca2-be15-56957df6cbfb None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:24.989870 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-16ba56f5-b423-4ca2-be15-56957df6cbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:25.001660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0e39f-36d5-42c0-a465-fe11c57615b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:25.002914 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0e39f-36d5-42c0-a465-fe11c57615b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:25.003216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0e39f-36d5-42c0-a465-fe11c57615b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:25.003503 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0e39f-36d5-42c0-a465-fe11c57615b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:25.004233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0e39f-36d5-42c0-a465-fe11c57615b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:25.006544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0e39f-36d5-42c0-a465-fe11c57615b9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:25.018118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1109/2238] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:25.022199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0e39f-36d5-42c0-a465-fe11c57615b9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:25.034510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0e39f-36d5-42c0-a465-fe11c57615b9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:25.051567 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a415719-ec91-43ba-a262-744558f9abc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:25.052586 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a415719-ec91-43ba-a262-744558f9abc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:25.052746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a415719-ec91-43ba-a262-744558f9abc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:25.053027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a415719-ec91-43ba-a262-744558f9abc6 admin admin] PATH_INFO: `/v3/projects/a53d3dafe3c8454188b83fdf000b8a5d/users/51beaf0b777043d1864c84dde454fa57/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:25.055776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a415719-ec91-43ba-a262-744558f9abc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a53d3dafe3c8454188b83fdf000b8a5d, user_id=51beaf0b777043d1864c84dde454fa57, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:25.059554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a415719-ec91-43ba-a262-744558f9abc6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:25.073967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba56f5-b423-4ca2-be15-56957df6cbfb None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:25.082822 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1110/2239] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:25 2022] PUT /identity/v3/projects/a53d3dafe3c8454188b83fdf000b8a5d/users/51beaf0b777043d1864c84dde454fa57/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:25.084489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a415719-ec91-43ba-a262-744558f9abc6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:25.089195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7a415719-ec91-43ba-a262-744558f9abc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:25.144319 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4ce45-48d6-493e-a50f-0ad952c84647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:25.145046 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4ce45-48d6-493e-a50f-0ad952c84647 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:25.146205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4ce45-48d6-493e-a50f-0ad952c84647 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:25.146205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4ce45-48d6-493e-a50f-0ad952c84647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:25.188068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a415719-ec91-43ba-a262-744558f9abc6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:25.194935 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6f8ec-f56c-4594-a497-4dc774e5789c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:25.196896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6f8ec-f56c-4594-a497-4dc774e5789c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:25.197192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6f8ec-f56c-4594-a497-4dc774e5789c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:25.197471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6f8ec-f56c-4594-a497-4dc774e5789c admin admin] PATH_INFO: `/v3/projects/d1d278a02d924d9f990bdf0bac334b05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:25.199314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6f8ec-f56c-4594-a497-4dc774e5789c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d278a02d924d9f990bdf0bac334b05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:25.202604 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6f8ec-f56c-4594-a497-4dc774e5789c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:25.241020 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4ce45-48d6-493e-a50f-0ad952c84647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:25.242489 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4ce45-48d6-493e-a50f-0ad952c84647 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:25.252383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1130/2240] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:20:25.253494 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dad121-5390-4c96-a336-7a2e209adb58 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:25.253777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dad121-5390-4c96-a336-7a2e209adb58 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:25.254057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dad121-5390-4c96-a336-7a2e209adb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:25.282847 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d5dad121-5390-4c96-a336-7a2e209adb58 None None] MFA Rules not processed for user `51beaf0b777043d1864c84dde454fa57`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:25.284430 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d5dad121-5390-4c96-a336-7a2e209adb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:25.355276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2ff6f8ec-f56c-4594-a497-4dc774e5789c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d278a02d924d9f990bdf0bac334b05', 'request_id': 'req-2ff6f8ec-f56c-4594-a497-4dc774e5789c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:25.366270 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2ff6f8ec-f56c-4594-a497-4dc774e5789c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d278a02d924d9f990bdf0bac334b05', 'request_id': 'req-2ff6f8ec-f56c-4594-a497-4dc774e5789c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:25.414726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1131/2241] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:25 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:25.422398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dad121-5390-4c96-a336-7a2e209adb58 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:25.424238 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d5dad121-5390-4c96-a336-7a2e209adb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:25.439544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dad121-5390-4c96-a336-7a2e209adb58 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:25.441986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2ff6f8ec-f56c-4594-a497-4dc774e5789c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d278a02d924d9f990bdf0bac334b05', 'request_id': 'req-2ff6f8ec-f56c-4594-a497-4dc774e5789c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:25.455854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2ff6f8ec-f56c-4594-a497-4dc774e5789c admin admin] The token cache is being invalidate because project d1d278a02d924d9f990bdf0bac334b05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:25.456490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2ff6f8ec-f56c-4594-a497-4dc774e5789c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:25.457852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2ff6f8ec-f56c-4594-a497-4dc774e5789c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:25.459355 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1111/2242] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:25 2022] DELETE /identity/v3/projects/d1d278a02d924d9f990bdf0bac334b05 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:25.463490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6f8ec-f56c-4594-a497-4dc774e5789c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:25.465225 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2ff6f8ec-f56c-4594-a497-4dc774e5789c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:25.492320 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd46a5-f434-48bf-a4f1-bfdd99355658 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:25.493193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd46a5-f434-48bf-a4f1-bfdd99355658 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:25.493537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd46a5-f434-48bf-a4f1-bfdd99355658 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:25.493962 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd46a5-f434-48bf-a4f1-bfdd99355658 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:25.495921 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c0bd46a5-f434-48bf-a4f1-bfdd99355658 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:25.534678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6f8ec-f56c-4594-a497-4dc774e5789c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:25.540004 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653764a-2782-4fb4-ae00-5d6d12f87241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:25.540893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653764a-2782-4fb4-ae00-5d6d12f87241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:25.541181 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653764a-2782-4fb4-ae00-5d6d12f87241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:25.541458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653764a-2782-4fb4-ae00-5d6d12f87241 admin admin] PATH_INFO: `/v3/users/3cc67573bb2b491b9c82b3657e5eed03` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:25.542673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653764a-2782-4fb4-ae00-5d6d12f87241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cc67573bb2b491b9c82b3657e5eed03)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:25.544859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653764a-2782-4fb4-ae00-5d6d12f87241 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:25.548531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:25.572579 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd46a5-f434-48bf-a4f1-bfdd99355658 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:25.574060 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd46a5-f434-48bf-a4f1-bfdd99355658 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:25.584411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1132/2243] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:25 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:20:25.753601 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a653764a-2782-4fb4-ae00-5d6d12f87241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cc67573bb2b491b9c82b3657e5eed03'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:25.765275 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a653764a-2782-4fb4-ae00-5d6d12f87241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cc67573bb2b491b9c82b3657e5eed03'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:25.765945 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a653764a-2782-4fb4-ae00-5d6d12f87241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cc67573bb2b491b9c82b3657e5eed03'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:25.773389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a653764a-2782-4fb4-ae00-5d6d12f87241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cc67573bb2b491b9c82b3657e5eed03'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:25.774019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a653764a-2782-4fb4-ae00-5d6d12f87241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cc67573bb2b491b9c82b3657e5eed03'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:25.848681 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:25.868440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd46a5-f434-48bf-a4f1-bfdd99355658 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:25.869914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c0bd46a5-f434-48bf-a4f1-bfdd99355658 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:25.882014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd46a5-f434-48bf-a4f1-bfdd99355658 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:25.886193 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683be238-b0e3-4acf-aa9e-5390fcbee3f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:25.886961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683be238-b0e3-4acf-aa9e-5390fcbee3f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:25.887351 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683be238-b0e3-4acf-aa9e-5390fcbee3f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:25.887717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683be238-b0e3-4acf-aa9e-5390fcbee3f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:25.889535 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-683be238-b0e3-4acf-aa9e-5390fcbee3f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:25.902932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1112/2244] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:25 2022] DELETE /identity/v3/users/3cc67573bb2b491b9c82b3657e5eed03 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:25.967829 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683be238-b0e3-4acf-aa9e-5390fcbee3f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:25.969426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683be238-b0e3-4acf-aa9e-5390fcbee3f2 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:25.982089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1133/2245] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:26.605999 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653764a-2782-4fb4-ae00-5d6d12f87241 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:26.608331 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a653764a-2782-4fb4-ae00-5d6d12f87241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:26.685443 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653764a-2782-4fb4-ae00-5d6d12f87241 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:26.691118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514486c2-a149-4a03-98b7-7003219d7ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:26.692039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514486c2-a149-4a03-98b7-7003219d7ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:26.692327 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514486c2-a149-4a03-98b7-7003219d7ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:26.692629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514486c2-a149-4a03-98b7-7003219d7ceb admin admin] PATH_INFO: `/v3/projects/12978f20d301496881b11bc3ba8bf5e2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:26.693967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514486c2-a149-4a03-98b7-7003219d7ceb admin admin] RBAC: Authorizing `identity:delete_project(project_id=12978f20d301496881b11bc3ba8bf5e2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:26.695914 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514486c2-a149-4a03-98b7-7003219d7ceb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:26.809921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-514486c2-a149-4a03-98b7-7003219d7ceb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12978f20d301496881b11bc3ba8bf5e2', 'request_id': 'req-514486c2-a149-4a03-98b7-7003219d7ceb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:26.817414 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-514486c2-a149-4a03-98b7-7003219d7ceb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12978f20d301496881b11bc3ba8bf5e2', 'request_id': 'req-514486c2-a149-4a03-98b7-7003219d7ceb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:26.871031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-514486c2-a149-4a03-98b7-7003219d7ceb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12978f20d301496881b11bc3ba8bf5e2', 'request_id': 'req-514486c2-a149-4a03-98b7-7003219d7ceb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:26.885168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-514486c2-a149-4a03-98b7-7003219d7ceb admin admin] The token cache is being invalidate because project 12978f20d301496881b11bc3ba8bf5e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:26.885459 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-514486c2-a149-4a03-98b7-7003219d7ceb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:26.886120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-514486c2-a149-4a03-98b7-7003219d7ceb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:26.887736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1113/2246] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:26 2022] DELETE /identity/v3/projects/12978f20d301496881b11bc3ba8bf5e2 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:26.894475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb5acd-1448-47c7-b16b-d3c4d3b12b87 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:26.894959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb5acd-1448-47c7-b16b-d3c4d3b12b87 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:26.895336 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb5acd-1448-47c7-b16b-d3c4d3b12b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:26.914450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6aeb5acd-1448-47c7-b16b-d3c4d3b12b87 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:26.915806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6aeb5acd-1448-47c7-b16b-d3c4d3b12b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:26.980344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1134/2247] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:26.988529 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fa6dc-f428-4ff5-8934-985b90477712 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:26.988813 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fa6dc-f428-4ff5-8934-985b90477712 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:26.989092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fa6dc-f428-4ff5-8934-985b90477712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:27.009081 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-871fa6dc-f428-4ff5-8934-985b90477712 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:27.009728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb5acd-1448-47c7-b16b-d3c4d3b12b87 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:27.010470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-871fa6dc-f428-4ff5-8934-985b90477712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:27.011121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6aeb5acd-1448-47c7-b16b-d3c4d3b12b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:27.020384 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1114/2248] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:27.024619 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871fa6dc-f428-4ff5-8934-985b90477712 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:27.026956 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb5acd-1448-47c7-b16b-d3c4d3b12b87 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:27.033811 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef47e11-283d-4604-b2e0-8c8e71f9eacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:27.035325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef47e11-283d-4604-b2e0-8c8e71f9eacf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:27.035435 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871fa6dc-f428-4ff5-8934-985b90477712 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:27.035748 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef47e11-283d-4604-b2e0-8c8e71f9eacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:27.036144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef47e11-283d-4604-b2e0-8c8e71f9eacf admin admin] PATH_INFO: `/v3/users/08e3a196d2074751823d4bb01c17c890` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:27.037366 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef47e11-283d-4604-b2e0-8c8e71f9eacf admin admin] RBAC: Authorizing `identity:delete_user(user_id=08e3a196d2074751823d4bb01c17c890)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:27.039572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef47e11-283d-4604-b2e0-8c8e71f9eacf admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:27.040172 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b75727-c05d-4d61-84bf-1ca5d34aa4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:27.040835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b75727-c05d-4d61-84bf-1ca5d34aa4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:27.043367 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b75727-c05d-4d61-84bf-1ca5d34aa4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:27.043367 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b75727-c05d-4d61-84bf-1ca5d34aa4c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:27.043367 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b75727-c05d-4d61-84bf-1ca5d34aa4c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:27.043367 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b75727-c05d-4d61-84bf-1ca5d34aa4c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:27.044093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b75727-c05d-4d61-84bf-1ca5d34aa4c1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:27.056154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1115/2249] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:20:27.060425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b75727-c05d-4d61-84bf-1ca5d34aa4c1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:27.070291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b75727-c05d-4d61-84bf-1ca5d34aa4c1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:27.074897 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14451a-d0d5-4542-80f3-5491accb8995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:27.075565 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14451a-d0d5-4542-80f3-5491accb8995 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:27.075854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14451a-d0d5-4542-80f3-5491accb8995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:27.076134 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14451a-d0d5-4542-80f3-5491accb8995 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:27.076819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14451a-d0d5-4542-80f3-5491accb8995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:27.078711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14451a-d0d5-4542-80f3-5491accb8995 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:27.189105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1116/2250] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:27 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:27.192734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14451a-d0d5-4542-80f3-5491accb8995 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:27.255934 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14451a-d0d5-4542-80f3-5491accb8995 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:27.258665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6ef47e11-283d-4604-b2e0-8c8e71f9eacf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08e3a196d2074751823d4bb01c17c890'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:27.262617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01eac70-0afb-4f3b-b649-ba29167b38fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:27.263519 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01eac70-0afb-4f3b-b649-ba29167b38fa admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:27.263810 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01eac70-0afb-4f3b-b649-ba29167b38fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:27.264088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01eac70-0afb-4f3b-b649-ba29167b38fa admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:27.265271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01eac70-0afb-4f3b-b649-ba29167b38fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:27.268630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01eac70-0afb-4f3b-b649-ba29167b38fa admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:27.274437 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6ef47e11-283d-4604-b2e0-8c8e71f9eacf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08e3a196d2074751823d4bb01c17c890'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:27.385858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1117/2251] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:27 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:27.389804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01eac70-0afb-4f3b-b649-ba29167b38fa admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:27.400975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01eac70-0afb-4f3b-b649-ba29167b38fa admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:27.406138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc95df-f170-4ddf-8998-54809e1f2f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:27.406807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc95df-f170-4ddf-8998-54809e1f2f66 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:27.407095 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc95df-f170-4ddf-8998-54809e1f2f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:27.408252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc95df-f170-4ddf-8998-54809e1f2f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:27.409244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc95df-f170-4ddf-8998-54809e1f2f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:27.410870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc95df-f170-4ddf-8998-54809e1f2f66 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:27.414845 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6ef47e11-283d-4604-b2e0-8c8e71f9eacf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08e3a196d2074751823d4bb01c17c890'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:27.415542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6ef47e11-283d-4604-b2e0-8c8e71f9eacf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08e3a196d2074751823d4bb01c17c890'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:27.421459 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1118/2252] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:27.423839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6ef47e11-283d-4604-b2e0-8c8e71f9eacf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08e3a196d2074751823d4bb01c17c890'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:27.425982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc95df-f170-4ddf-8998-54809e1f2f66 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:27.428479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-96bc95df-f170-4ddf-8998-54809e1f2f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:27.430537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1135/2253] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:27 2022] DELETE /identity/v3/users/08e3a196d2074751823d4bb01c17c890 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:27.434962 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc58e1f-a68f-4fc2-a458-574c091b8831 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:27.435297 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc58e1f-a68f-4fc2-a458-574c091b8831 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:27.435607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc58e1f-a68f-4fc2-a458-574c091b8831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:27.444384 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc95df-f170-4ddf-8998-54809e1f2f66 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:27.453606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bdc58e1f-a68f-4fc2-a458-574c091b8831 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:27.455751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bdc58e1f-a68f-4fc2-a458-574c091b8831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:27.502677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0236b-4b9f-47a9-9cb8-d4fc44825d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:27.503501 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0236b-4b9f-47a9-9cb8-d4fc44825d1b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:27.503789 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0236b-4b9f-47a9-9cb8-d4fc44825d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:27.504070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0236b-4b9f-47a9-9cb8-d4fc44825d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:27.504923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0236b-4b9f-47a9-9cb8-d4fc44825d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:27.506663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0236b-4b9f-47a9-9cb8-d4fc44825d1b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:27.516118 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1136/2254] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:27.520717 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1119/2255] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:27.522473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc58e1f-a68f-4fc2-a458-574c091b8831 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:27.526330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0236b-4b9f-47a9-9cb8-d4fc44825d1b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:27.528150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bfd0236b-4b9f-47a9-9cb8-d4fc44825d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:27.532494 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc58e1f-a68f-4fc2-a458-574c091b8831 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:27.537436 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5635e7f-42ad-4d69-8cd3-3a489a48a186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:27.538317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5635e7f-42ad-4d69-8cd3-3a489a48a186 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:27.538597 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5635e7f-42ad-4d69-8cd3-3a489a48a186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:27.538871 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5635e7f-42ad-4d69-8cd3-3a489a48a186 admin admin] PATH_INFO: `/v3/projects/4a4a374fe5f64ff6ba0b37f62d2d5f4d/users/be6528b7fcfc4cdd914d3d86ef86dff9/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:27.555428 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5635e7f-42ad-4d69-8cd3-3a489a48a186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4a374fe5f64ff6ba0b37f62d2d5f4d, user_id=be6528b7fcfc4cdd914d3d86ef86dff9, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:27.558112 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5635e7f-42ad-4d69-8cd3-3a489a48a186 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:27.561920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:27.606881 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0236b-4b9f-47a9-9cb8-d4fc44825d1b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:27.613592 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0531b68-c5d6-4c22-a0a6-e2724af765c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:27.614642 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0531b68-c5d6-4c22-a0a6-e2724af765c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:27.615126 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0531b68-c5d6-4c22-a0a6-e2724af765c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:27.615635 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0531b68-c5d6-4c22-a0a6-e2724af765c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:27.631469 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0531b68-c5d6-4c22-a0a6-e2724af765c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:27.634447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0531b68-c5d6-4c22-a0a6-e2724af765c5 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:27.638887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1137/2256] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:27 2022] PUT /identity/v3/projects/4a4a374fe5f64ff6ba0b37f62d2d5f4d/users/be6528b7fcfc4cdd914d3d86ef86dff9/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:27.651220 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2ef73-1004-4d1a-80f2-546d60771124 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:27.651514 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2ef73-1004-4d1a-80f2-546d60771124 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:27.651827 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2ef73-1004-4d1a-80f2-546d60771124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:27.652711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1120/2257] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:20:27.680613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-84c2ef73-1004-4d1a-80f2-546d60771124 None None] MFA Rules not processed for user `be6528b7fcfc4cdd914d3d86ef86dff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:27.681875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-84c2ef73-1004-4d1a-80f2-546d60771124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:27.774313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1138/2258] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:27.790970 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0531b68-c5d6-4c22-a0a6-e2724af765c5 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:27.790970 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b0531b68-c5d6-4c22-a0a6-e2724af765c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:27.817827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:27.851223 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0531b68-c5d6-4c22-a0a6-e2724af765c5 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:27.913976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e103b-be8f-46fe-a2d1-b4e895a15b84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:27.914721 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e103b-be8f-46fe-a2d1-b4e895a15b84 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:27.915015 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e103b-be8f-46fe-a2d1-b4e895a15b84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:27.915344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e103b-be8f-46fe-a2d1-b4e895a15b84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:27.926711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e103b-be8f-46fe-a2d1-b4e895a15b84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:27.928292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e103b-be8f-46fe-a2d1-b4e895a15b84 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:27.941047 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1121/2259] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:28.140438 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2ef73-1004-4d1a-80f2-546d60771124 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:28.142097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-84c2ef73-1004-4d1a-80f2-546d60771124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:28.216838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2ef73-1004-4d1a-80f2-546d60771124 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:28.222860 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:28.223787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:28.224076 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:28.224392 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3 admin admin] PATH_INFO: `/v3/projects/8576fbbfaba749afa20fe3a4305a6aa0` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:28.225576 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8576fbbfaba749afa20fe3a4305a6aa0)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:28.227520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:28.375363 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8576fbbfaba749afa20fe3a4305a6aa0', 'request_id': 'req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:28.382161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8576fbbfaba749afa20fe3a4305a6aa0', 'request_id': 'req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:28.493680 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8576fbbfaba749afa20fe3a4305a6aa0', 'request_id': 'req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:28.506321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3 admin admin] The token cache is being invalidate because project 8576fbbfaba749afa20fe3a4305a6aa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:28.506608 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:28.507508 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:28.509140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1139/2260] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:28 2022] DELETE /identity/v3/projects/8576fbbfaba749afa20fe3a4305a6aa0 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:28.513223 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e103b-be8f-46fe-a2d1-b4e895a15b84 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:28.526337 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e103b-be8f-46fe-a2d1-b4e895a15b84 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:28.585785 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53ae41-0db4-416b-8dc3-72f8fdeefdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:28.586669 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53ae41-0db4-416b-8dc3-72f8fdeefdc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:28.586955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53ae41-0db4-416b-8dc3-72f8fdeefdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:28.587266 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53ae41-0db4-416b-8dc3-72f8fdeefdc1 admin admin] PATH_INFO: `/v3/users/dc6274e0ee844f09a9e5a730ed0f6ead` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:28.588480 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53ae41-0db4-416b-8dc3-72f8fdeefdc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc6274e0ee844f09a9e5a730ed0f6ead)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:28.590595 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53ae41-0db4-416b-8dc3-72f8fdeefdc1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:28.801505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6e53ae41-0db4-416b-8dc3-72f8fdeefdc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc6274e0ee844f09a9e5a730ed0f6ead'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:28.813987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6e53ae41-0db4-416b-8dc3-72f8fdeefdc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc6274e0ee844f09a9e5a730ed0f6ead'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:28.818824 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6e53ae41-0db4-416b-8dc3-72f8fdeefdc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc6274e0ee844f09a9e5a730ed0f6ead'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:28.821192 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:28.822177 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6e53ae41-0db4-416b-8dc3-72f8fdeefdc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc6274e0ee844f09a9e5a730ed0f6ead'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:28.822842 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6e53ae41-0db4-416b-8dc3-72f8fdeefdc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc6274e0ee844f09a9e5a730ed0f6ead'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:28.906917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1122/2261] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:28 2022] DELETE /identity/v3/users/dc6274e0ee844f09a9e5a730ed0f6ead => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:29.238277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:29.342403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:29.343868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:29.357696 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc1af0-07fb-4a98-95c5-ef9bec0357d3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:29.414690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ff23e-5231-4875-8871-ea9994f5d100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:29.415520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ff23e-5231-4875-8871-ea9994f5d100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:29.415806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ff23e-5231-4875-8871-ea9994f5d100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:29.416084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ff23e-5231-4875-8871-ea9994f5d100 admin admin] PATH_INFO: `/v3/projects/1047d27e59074281b2dd61dbfe910beb` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:29.417260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ff23e-5231-4875-8871-ea9994f5d100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1047d27e59074281b2dd61dbfe910beb)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:29.419395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ff23e-5231-4875-8871-ea9994f5d100 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:29.545033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-357ff23e-5231-4875-8871-ea9994f5d100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1047d27e59074281b2dd61dbfe910beb', 'request_id': 'req-357ff23e-5231-4875-8871-ea9994f5d100'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:29.548118 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-357ff23e-5231-4875-8871-ea9994f5d100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1047d27e59074281b2dd61dbfe910beb', 'request_id': 'req-357ff23e-5231-4875-8871-ea9994f5d100'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:29.620363 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-357ff23e-5231-4875-8871-ea9994f5d100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1047d27e59074281b2dd61dbfe910beb', 'request_id': 'req-357ff23e-5231-4875-8871-ea9994f5d100'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:29.634366 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-357ff23e-5231-4875-8871-ea9994f5d100 admin admin] The token cache is being invalidate because project 1047d27e59074281b2dd61dbfe910beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:29.634839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-357ff23e-5231-4875-8871-ea9994f5d100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:29.635847 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-357ff23e-5231-4875-8871-ea9994f5d100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:29.641238 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1140/2262] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:29 2022] DELETE /identity/v3/projects/1047d27e59074281b2dd61dbfe910beb => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:29.643963 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1e2d2-d1f8-46a3-ac88-c1fd21a64797 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:29.644546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1e2d2-d1f8-46a3-ac88-c1fd21a64797 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:29.645017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1e2d2-d1f8-46a3-ac88-c1fd21a64797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:29.671284 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-55d1e2d2-d1f8-46a3-ac88-c1fd21a64797 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:29.671284 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-55d1e2d2-d1f8-46a3-ac88-c1fd21a64797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:29.740306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1123/2263] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:29.747837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2afd4-cc87-4e44-8ced-1e0ef7665480 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:29.748560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2afd4-cc87-4e44-8ced-1e0ef7665480 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:29.748850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2afd4-cc87-4e44-8ced-1e0ef7665480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:29.768306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c8d2afd4-cc87-4e44-8ced-1e0ef7665480 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:29.769529 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c8d2afd4-cc87-4e44-8ced-1e0ef7665480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:29.782822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1141/2264] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:29.787161 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1e2d2-d1f8-46a3-ac88-c1fd21a64797 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:29.796784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1e2d2-d1f8-46a3-ac88-c1fd21a64797 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:29.801914 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4b6bb-0948-4efc-963e-8a7dcd312c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:29.802572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4b6bb-0948-4efc-963e-8a7dcd312c72 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:29.802860 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4b6bb-0948-4efc-963e-8a7dcd312c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:29.808678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4b6bb-0948-4efc-963e-8a7dcd312c72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:29.808678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4b6bb-0948-4efc-963e-8a7dcd312c72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:29.808678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4b6bb-0948-4efc-963e-8a7dcd312c72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:29.808678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4b6bb-0948-4efc-963e-8a7dcd312c72 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:29.818389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1124/2265] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:20:29.822842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2afd4-cc87-4e44-8ced-1e0ef7665480 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:29.832847 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2afd4-cc87-4e44-8ced-1e0ef7665480 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:29.838056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f887d-6840-439e-a2a3-798bfc2dcdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:29.838785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f887d-6840-439e-a2a3-798bfc2dcdb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:29.839096 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f887d-6840-439e-a2a3-798bfc2dcdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:29.839421 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f887d-6840-439e-a2a3-798bfc2dcdb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:29.840197 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f887d-6840-439e-a2a3-798bfc2dcdb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:29.842272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f887d-6840-439e-a2a3-798bfc2dcdb2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:29.857365 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:29.931422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1142/2266] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:29 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:29.935376 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4b6bb-0948-4efc-963e-8a7dcd312c72 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:29.946220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4b6bb-0948-4efc-963e-8a7dcd312c72 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:30.004221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252c808-2465-4a11-b003-cf6dfd85ee45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:30.005031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252c808-2465-4a11-b003-cf6dfd85ee45 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:30.005324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252c808-2465-4a11-b003-cf6dfd85ee45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:30.005614 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252c808-2465-4a11-b003-cf6dfd85ee45 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:30.006303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252c808-2465-4a11-b003-cf6dfd85ee45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:30.008304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252c808-2465-4a11-b003-cf6dfd85ee45 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:30.129752 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1125/2267] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:29 2022] POST /identity/v3/users => generated 284 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:30.133364 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f887d-6840-439e-a2a3-798bfc2dcdb2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:30.135633 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:30.145838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f887d-6840-439e-a2a3-798bfc2dcdb2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:30.150589 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2d39b-19eb-4dd6-9860-778ee8abfba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:30.151281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2d39b-19eb-4dd6-9860-778ee8abfba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:30.151569 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2d39b-19eb-4dd6-9860-778ee8abfba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:30.151848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2d39b-19eb-4dd6-9860-778ee8abfba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:30.152575 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2d39b-19eb-4dd6-9860-778ee8abfba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:30.154426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2d39b-19eb-4dd6-9860-778ee8abfba0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:30.165406 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1143/2268] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:30.169509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252c808-2465-4a11-b003-cf6dfd85ee45 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:30.180024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252c808-2465-4a11-b003-cf6dfd85ee45 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:30.184728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e585272-26d1-461e-ba4d-3f3ee7d91964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:30.185333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e585272-26d1-461e-ba4d-3f3ee7d91964 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:30.185614 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e585272-26d1-461e-ba4d-3f3ee7d91964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:30.185890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e585272-26d1-461e-ba4d-3f3ee7d91964 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:30.186607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e585272-26d1-461e-ba4d-3f3ee7d91964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:30.188267 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e585272-26d1-461e-ba4d-3f3ee7d91964 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:30.199401 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1126/2269] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:30.203517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2d39b-19eb-4dd6-9860-778ee8abfba0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:30.213429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2d39b-19eb-4dd6-9860-778ee8abfba0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:30.218095 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b30d4-e619-46a1-ac18-68bdfd6c4fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:30.218971 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b30d4-e619-46a1-ac18-68bdfd6c4fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:30.219289 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b30d4-e619-46a1-ac18-68bdfd6c4fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:30.223543 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b30d4-e619-46a1-ac18-68bdfd6c4fd3 admin admin] PATH_INFO: `/v3/projects/aec47099c2e74ba3a1d5cf86fb6456f5/users/905186a697e44d77b94fc2bb20302b12/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:30.234126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b30d4-e619-46a1-ac18-68bdfd6c4fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aec47099c2e74ba3a1d5cf86fb6456f5, user_id=905186a697e44d77b94fc2bb20302b12, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:30.237437 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b30d4-e619-46a1-ac18-68bdfd6c4fd3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:30.317340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1144/2270] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:30 2022] PUT /identity/v3/projects/aec47099c2e74ba3a1d5cf86fb6456f5/users/905186a697e44d77b94fc2bb20302b12/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:30.323321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:30.327553 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cd9db-dc12-4672-a6cf-cb36230fcf03 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:30.327843 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cd9db-dc12-4672-a6cf-cb36230fcf03 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:30.328155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cd9db-dc12-4672-a6cf-cb36230fcf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:30.355963 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-440cd9db-dc12-4672-a6cf-cb36230fcf03 None None] MFA Rules not processed for user `905186a697e44d77b94fc2bb20302b12`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:30.357182 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-440cd9db-dc12-4672-a6cf-cb36230fcf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:30.421184 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1127/2271] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:30.426471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b30d4-e619-46a1-ac18-68bdfd6c4fd3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:30.487410 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b30d4-e619-46a1-ac18-68bdfd6c4fd3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:30.491897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c54bf1-bc28-436e-8784-4aaa0c91d4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:30.492664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c54bf1-bc28-436e-8784-4aaa0c91d4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:30.492950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c54bf1-bc28-436e-8784-4aaa0c91d4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:30.493224 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c54bf1-bc28-436e-8784-4aaa0c91d4f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:30.493917 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c54bf1-bc28-436e-8784-4aaa0c91d4f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:30.495848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c54bf1-bc28-436e-8784-4aaa0c91d4f1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:30.620061 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1145/2272] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:30 2022] POST /identity/v3/projects => generated 367 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:30.624158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cd9db-dc12-4672-a6cf-cb36230fcf03 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:30.635485 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cd9db-dc12-4672-a6cf-cb36230fcf03 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:30.690712 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb352a0a-e2b9-4a94-b1cf-6319956ac98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:30.691412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb352a0a-e2b9-4a94-b1cf-6319956ac98f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:30.691702 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb352a0a-e2b9-4a94-b1cf-6319956ac98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:30.691980 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb352a0a-e2b9-4a94-b1cf-6319956ac98f admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:30.692676 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb352a0a-e2b9-4a94-b1cf-6319956ac98f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:30.694582 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb352a0a-e2b9-4a94-b1cf-6319956ac98f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:30.770159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1128/2273] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:30 2022] POST /identity/v3/users => generated 290 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:30.773946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c54bf1-bc28-436e-8784-4aaa0c91d4f1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:30.784883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c54bf1-bc28-436e-8784-4aaa0c91d4f1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:30.791057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebf232-a6ec-43eb-b57a-298d1824b36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:30.791057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebf232-a6ec-43eb-b57a-298d1824b36f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:30.791057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebf232-a6ec-43eb-b57a-298d1824b36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:30.791057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebf232-a6ec-43eb-b57a-298d1824b36f admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:30.791995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebf232-a6ec-43eb-b57a-298d1824b36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:30.793253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebf232-a6ec-43eb-b57a-298d1824b36f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:30.806778 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1146/2274] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:30.815393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb352a0a-e2b9-4a94-b1cf-6319956ac98f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:30.827895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:30.836213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb352a0a-e2b9-4a94-b1cf-6319956ac98f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:30.851539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:30.861600 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af052a12-c46e-46a1-b0e9-63f5e51b6ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:30.864421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af052a12-c46e-46a1-b0e9-63f5e51b6ecf admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:30.866687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af052a12-c46e-46a1-b0e9-63f5e51b6ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:30.867323 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af052a12-c46e-46a1-b0e9-63f5e51b6ecf admin admin] PATH_INFO: `/v3/projects/ada77210f6f04fb1bf376da287ff7236/users/2f2949e42d2f4c4189912f55ea4bfc5e/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:30.897988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af052a12-c46e-46a1-b0e9-63f5e51b6ecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ada77210f6f04fb1bf376da287ff7236, user_id=2f2949e42d2f4c4189912f55ea4bfc5e, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:30.898511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af052a12-c46e-46a1-b0e9-63f5e51b6ecf admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:31.009749 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1129/2275] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:30 2022] PUT /identity/v3/projects/ada77210f6f04fb1bf376da287ff7236/users/2f2949e42d2f4c4189912f55ea4bfc5e/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:31.013798 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebf232-a6ec-43eb-b57a-298d1824b36f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:31.026181 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebf232-a6ec-43eb-b57a-298d1824b36f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:31.084787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01d380-14b7-4790-a260-3390e7609101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:31.086771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01d380-14b7-4790-a260-3390e7609101 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:31.086771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01d380-14b7-4790-a260-3390e7609101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:31.086771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01d380-14b7-4790-a260-3390e7609101 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:31.086771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01d380-14b7-4790-a260-3390e7609101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:31.096103 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01d380-14b7-4790-a260-3390e7609101 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:31.100428 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1147/2276] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:31.106193 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af052a12-c46e-46a1-b0e9-63f5e51b6ecf admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:31.118238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af052a12-c46e-46a1-b0e9-63f5e51b6ecf admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:31.123543 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be227557-5e25-4c67-9d31-203c8c54f4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:31.124515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be227557-5e25-4c67-9d31-203c8c54f4aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:31.124870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be227557-5e25-4c67-9d31-203c8c54f4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:31.125221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be227557-5e25-4c67-9d31-203c8c54f4aa admin admin] PATH_INFO: `/v3/projects/ada77210f6f04fb1bf376da287ff7236/users/2f2949e42d2f4c4189912f55ea4bfc5e/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:31.127734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be227557-5e25-4c67-9d31-203c8c54f4aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ada77210f6f04fb1bf376da287ff7236, user_id=2f2949e42d2f4c4189912f55ea4bfc5e, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:31.133528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be227557-5e25-4c67-9d31-203c8c54f4aa admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:31.150519 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1130/2277] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:31 2022] PUT /identity/v3/projects/ada77210f6f04fb1bf376da287ff7236/users/2f2949e42d2f4c4189912f55ea4bfc5e/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:31.157214 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3307e0c-026e-4cf8-ad68-dd6a80cebd6c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:31.157507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3307e0c-026e-4cf8-ad68-dd6a80cebd6c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:31.158310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3307e0c-026e-4cf8-ad68-dd6a80cebd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:31.187872 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c3307e0c-026e-4cf8-ad68-dd6a80cebd6c None None] MFA Rules not processed for user `2f2949e42d2f4c4189912f55ea4bfc5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:31.190433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c3307e0c-026e-4cf8-ad68-dd6a80cebd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:31.260420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1148/2278] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:31 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:31.272230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be227557-5e25-4c67-9d31-203c8c54f4aa admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:31.274483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-be227557-5e25-4c67-9d31-203c8c54f4aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:31.334485 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be227557-5e25-4c67-9d31-203c8c54f4aa admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:31.338374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5adc9b1-793f-4b44-a0af-051fa226a7f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:31.339020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5adc9b1-793f-4b44-a0af-051fa226a7f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:31.339340 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5adc9b1-793f-4b44-a0af-051fa226a7f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:31.339615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5adc9b1-793f-4b44-a0af-051fa226a7f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:31.346605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:31.348836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5adc9b1-793f-4b44-a0af-051fa226a7f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:31.350429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5adc9b1-793f-4b44-a0af-051fa226a7f6 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:31.359790 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1131/2279] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:31 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:31.552013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:31.571505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3307e0c-026e-4cf8-ad68-dd6a80cebd6c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:31.581061 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3307e0c-026e-4cf8-ad68-dd6a80cebd6c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:31.584693 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b83692-20d6-43dc-8db3-be7b13c8b5c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:31.585322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b83692-20d6-43dc-8db3-be7b13c8b5c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:31.590804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b83692-20d6-43dc-8db3-be7b13c8b5c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:31.590804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b83692-20d6-43dc-8db3-be7b13c8b5c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:31.642296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b83692-20d6-43dc-8db3-be7b13c8b5c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:31.644029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b83692-20d6-43dc-8db3-be7b13c8b5c1 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:31.653031 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1149/2280] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:31.755893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:31.832256 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:32.512123 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:32.699631 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5adc9b1-793f-4b44-a0af-051fa226a7f6 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:32.701052 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c5adc9b1-793f-4b44-a0af-051fa226a7f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:32.770902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5adc9b1-793f-4b44-a0af-051fa226a7f6 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:32.775486 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3043189-49cd-4de4-a406-6b42f2ab7fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:32.776370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3043189-49cd-4de4-a406-6b42f2ab7fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:32.776657 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3043189-49cd-4de4-a406-6b42f2ab7fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:32.776938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3043189-49cd-4de4-a406-6b42f2ab7fa5 admin admin] PATH_INFO: `/v3/users/f3fea01118f54873a5f59d6c22b74205` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:32.778202 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3043189-49cd-4de4-a406-6b42f2ab7fa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3fea01118f54873a5f59d6c22b74205)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:32.780420 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3043189-49cd-4de4-a406-6b42f2ab7fa5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:32.964742 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b3043189-49cd-4de4-a406-6b42f2ab7fa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3fea01118f54873a5f59d6c22b74205'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:32.977034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b3043189-49cd-4de4-a406-6b42f2ab7fa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3fea01118f54873a5f59d6c22b74205'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:32.977724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b3043189-49cd-4de4-a406-6b42f2ab7fa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3fea01118f54873a5f59d6c22b74205'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:32.985181 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b3043189-49cd-4de4-a406-6b42f2ab7fa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3fea01118f54873a5f59d6c22b74205'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:32.986323 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b3043189-49cd-4de4-a406-6b42f2ab7fa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3fea01118f54873a5f59d6c22b74205'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:33.072320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1132/2281] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:32 2022] DELETE /identity/v3/users/f3fea01118f54873a5f59d6c22b74205 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:33.076981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f4c0c-8b05-41d3-a643-86d9c68834c1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:33.077454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f4c0c-8b05-41d3-a643-86d9c68834c1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:33.077547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f4c0c-8b05-41d3-a643-86d9c68834c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:33.096092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4b7f4c0c-8b05-41d3-a643-86d9c68834c1 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:33.097311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4b7f4c0c-8b05-41d3-a643-86d9c68834c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:33.160515 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1150/2282] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:33.170423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3043189-49cd-4de4-a406-6b42f2ab7fa5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:33.172024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b3043189-49cd-4de4-a406-6b42f2ab7fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:33.233808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3043189-49cd-4de4-a406-6b42f2ab7fa5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:33.238723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48530024-cc95-475d-a22b-4ace22a2cf9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:33.239436 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48530024-cc95-475d-a22b-4ace22a2cf9c service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:33.239720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48530024-cc95-475d-a22b-4ace22a2cf9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:33.240073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48530024-cc95-475d-a22b-4ace22a2cf9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:33.249206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48530024-cc95-475d-a22b-4ace22a2cf9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:33.250665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48530024-cc95-475d-a22b-4ace22a2cf9c service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:33.260444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1133/2283] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:33.769899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f4c0c-8b05-41d3-a643-86d9c68834c1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:33.771452 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4b7f4c0c-8b05-41d3-a643-86d9c68834c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:33.784832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f4c0c-8b05-41d3-a643-86d9c68834c1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:33.789453 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a16bec-bec5-416f-81f2-ea524ac929ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:33.790483 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a16bec-bec5-416f-81f2-ea524ac929ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:33.790771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a16bec-bec5-416f-81f2-ea524ac929ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:33.791056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a16bec-bec5-416f-81f2-ea524ac929ef admin admin] PATH_INFO: `/v3/projects/3466295427d947bb901d3e41e25c0581` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:33.792340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a16bec-bec5-416f-81f2-ea524ac929ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=3466295427d947bb901d3e41e25c0581)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:33.794259 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a16bec-bec5-416f-81f2-ea524ac929ef admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:33.905139 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:33.908211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-64a16bec-bec5-416f-81f2-ea524ac929ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3466295427d947bb901d3e41e25c0581', 'request_id': 'req-64a16bec-bec5-416f-81f2-ea524ac929ef'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:33.914375 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-64a16bec-bec5-416f-81f2-ea524ac929ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3466295427d947bb901d3e41e25c0581', 'request_id': 'req-64a16bec-bec5-416f-81f2-ea524ac929ef'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:33.986662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-64a16bec-bec5-416f-81f2-ea524ac929ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3466295427d947bb901d3e41e25c0581', 'request_id': 'req-64a16bec-bec5-416f-81f2-ea524ac929ef'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:33.997962 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-64a16bec-bec5-416f-81f2-ea524ac929ef admin admin] The token cache is being invalidate because project 3466295427d947bb901d3e41e25c0581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:33.998258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-64a16bec-bec5-416f-81f2-ea524ac929ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:33.998915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-64a16bec-bec5-416f-81f2-ea524ac929ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:34.000487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1151/2284] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:33 2022] DELETE /identity/v3/projects/3466295427d947bb901d3e41e25c0581 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:34.004989 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48530024-cc95-475d-a22b-4ace22a2cf9c service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:34.014909 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48530024-cc95-475d-a22b-4ace22a2cf9c service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:34.026360 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:34.072025 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da69ae-9fa3-4364-b6c6-4422484994df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:34.072931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da69ae-9fa3-4364-b6c6-4422484994df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:34.073168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da69ae-9fa3-4364-b6c6-4422484994df admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:34.073440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da69ae-9fa3-4364-b6c6-4422484994df admin admin] PATH_INFO: `/v3/users/51beaf0b777043d1864c84dde454fa57` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:34.074624 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da69ae-9fa3-4364-b6c6-4422484994df admin admin] RBAC: Authorizing `identity:delete_user(user_id=51beaf0b777043d1864c84dde454fa57)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:34.076575 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da69ae-9fa3-4364-b6c6-4422484994df admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:34.369695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-03da69ae-9fa3-4364-b6c6-4422484994df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51beaf0b777043d1864c84dde454fa57'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:34.381994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-03da69ae-9fa3-4364-b6c6-4422484994df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51beaf0b777043d1864c84dde454fa57'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:34.382685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-03da69ae-9fa3-4364-b6c6-4422484994df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51beaf0b777043d1864c84dde454fa57'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:34.389394 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-03da69ae-9fa3-4364-b6c6-4422484994df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51beaf0b777043d1864c84dde454fa57'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:34.390033 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-03da69ae-9fa3-4364-b6c6-4422484994df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51beaf0b777043d1864c84dde454fa57'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:34.488745 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1134/2285] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:34 2022] DELETE /identity/v3/users/51beaf0b777043d1864c84dde454fa57 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:34.889963 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a16bec-bec5-416f-81f2-ea524ac929ef admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:34.891447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-64a16bec-bec5-416f-81f2-ea524ac929ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:34.905426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a16bec-bec5-416f-81f2-ea524ac929ef admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:34.967482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3ae59-eee6-4f9a-beed-0c81d5708540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:34.968280 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3ae59-eee6-4f9a-beed-0c81d5708540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:34.968568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3ae59-eee6-4f9a-beed-0c81d5708540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:34.968841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3ae59-eee6-4f9a-beed-0c81d5708540 admin admin] PATH_INFO: `/v3/projects/a53d3dafe3c8454188b83fdf000b8a5d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:34.970043 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3ae59-eee6-4f9a-beed-0c81d5708540 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a53d3dafe3c8454188b83fdf000b8a5d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:34.972819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3ae59-eee6-4f9a-beed-0c81d5708540 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:35.082811 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c7f3ae59-eee6-4f9a-beed-0c81d5708540 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a53d3dafe3c8454188b83fdf000b8a5d', 'request_id': 'req-c7f3ae59-eee6-4f9a-beed-0c81d5708540'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:35.088944 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c7f3ae59-eee6-4f9a-beed-0c81d5708540 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a53d3dafe3c8454188b83fdf000b8a5d', 'request_id': 'req-c7f3ae59-eee6-4f9a-beed-0c81d5708540'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:35.145700 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c7f3ae59-eee6-4f9a-beed-0c81d5708540 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a53d3dafe3c8454188b83fdf000b8a5d', 'request_id': 'req-c7f3ae59-eee6-4f9a-beed-0c81d5708540'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:35.156795 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c7f3ae59-eee6-4f9a-beed-0c81d5708540 admin admin] The token cache is being invalidate because project a53d3dafe3c8454188b83fdf000b8a5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:35.157083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c7f3ae59-eee6-4f9a-beed-0c81d5708540 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:35.158461 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c7f3ae59-eee6-4f9a-beed-0c81d5708540 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:35.160069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1152/2286] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:34 2022] DELETE /identity/v3/projects/a53d3dafe3c8454188b83fdf000b8a5d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:35.164956 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4918f-8c4c-4ed5-9a77-9d471356bc35 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:35.165435 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4918f-8c4c-4ed5-9a77-9d471356bc35 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:35.165523 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4918f-8c4c-4ed5-9a77-9d471356bc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:35.184219 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-89c4918f-8c4c-4ed5-9a77-9d471356bc35 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:35.185418 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-89c4918f-8c4c-4ed5-9a77-9d471356bc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:35.245735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1135/2287] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:35.253347 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94768937-6e3b-4425-9ba8-050077db6548 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:35.253639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94768937-6e3b-4425-9ba8-050077db6548 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:35.253919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94768937-6e3b-4425-9ba8-050077db6548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:35.272538 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-94768937-6e3b-4425-9ba8-050077db6548 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:35.273737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-94768937-6e3b-4425-9ba8-050077db6548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:35.281988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1153/2288] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:35.286657 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4918f-8c4c-4ed5-9a77-9d471356bc35 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:35.296064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4918f-8c4c-4ed5-9a77-9d471356bc35 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:35.300697 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37e273-710c-49af-bf80-0ff989550c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:35.301344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37e273-710c-49af-bf80-0ff989550c8a admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:35.301625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37e273-710c-49af-bf80-0ff989550c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:35.301900 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37e273-710c-49af-bf80-0ff989550c8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:35.302616 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37e273-710c-49af-bf80-0ff989550c8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:35.302923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37e273-710c-49af-bf80-0ff989550c8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:35.304541 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37e273-710c-49af-bf80-0ff989550c8a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:35.315736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1136/2289] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:20:35.319831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94768937-6e3b-4425-9ba8-050077db6548 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:35.329015 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94768937-6e3b-4425-9ba8-050077db6548 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:35.334358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571702a-1749-46ed-80b8-6cb76d0baa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:35.335004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571702a-1749-46ed-80b8-6cb76d0baa62 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:35.335389 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571702a-1749-46ed-80b8-6cb76d0baa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:35.335673 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571702a-1749-46ed-80b8-6cb76d0baa62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:35.336461 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571702a-1749-46ed-80b8-6cb76d0baa62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:35.338361 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571702a-1749-46ed-80b8-6cb76d0baa62 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:35.426886 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1154/2290] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:35 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:35.430844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37e273-710c-49af-bf80-0ff989550c8a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:35.441350 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37e273-710c-49af-bf80-0ff989550c8a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:35.461492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:35.498238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff74796-eb02-4733-982b-47948bd25385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:35.498910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff74796-eb02-4733-982b-47948bd25385 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:35.499209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff74796-eb02-4733-982b-47948bd25385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:35.504001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff74796-eb02-4733-982b-47948bd25385 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:35.504001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff74796-eb02-4733-982b-47948bd25385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:35.504001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff74796-eb02-4733-982b-47948bd25385 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:35.615695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1137/2291] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:35 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:35.621185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571702a-1749-46ed-80b8-6cb76d0baa62 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:35.632589 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571702a-1749-46ed-80b8-6cb76d0baa62 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:35.637980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490108ad-d3a9-43f4-ac6e-8b7e68e9bbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:35.638658 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490108ad-d3a9-43f4-ac6e-8b7e68e9bbac admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:35.642958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490108ad-d3a9-43f4-ac6e-8b7e68e9bbac admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:35.642958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490108ad-d3a9-43f4-ac6e-8b7e68e9bbac admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:35.642958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490108ad-d3a9-43f4-ac6e-8b7e68e9bbac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:35.642958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490108ad-d3a9-43f4-ac6e-8b7e68e9bbac admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:35.655240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1155/2292] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:35.658312 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff74796-eb02-4733-982b-47948bd25385 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:35.668177 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff74796-eb02-4733-982b-47948bd25385 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:35.672722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c95d26-1075-4f1f-a47b-c47ec1a74743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:35.673435 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c95d26-1075-4f1f-a47b-c47ec1a74743 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:35.673714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c95d26-1075-4f1f-a47b-c47ec1a74743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:35.679962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c95d26-1075-4f1f-a47b-c47ec1a74743 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:35.679962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c95d26-1075-4f1f-a47b-c47ec1a74743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:35.679962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c95d26-1075-4f1f-a47b-c47ec1a74743 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:35.686560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1138/2293] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:35.691298 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490108ad-d3a9-43f4-ac6e-8b7e68e9bbac admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:35.701696 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490108ad-d3a9-43f4-ac6e-8b7e68e9bbac admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:35.707738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223d26e-43b5-4e66-8b0e-7c3773c23d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:35.708628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223d26e-43b5-4e66-8b0e-7c3773c23d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:35.708908 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223d26e-43b5-4e66-8b0e-7c3773c23d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:35.709187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223d26e-43b5-4e66-8b0e-7c3773c23d8e admin admin] PATH_INFO: `/v3/projects/926c7a8ed8c3437c99cc0afaf36e5ae7/users/4fd8b0ec0f584c6ea0c8b6be042b8f17/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:35.722882 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:35.726045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223d26e-43b5-4e66-8b0e-7c3773c23d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=926c7a8ed8c3437c99cc0afaf36e5ae7, user_id=4fd8b0ec0f584c6ea0c8b6be042b8f17, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:35.728824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223d26e-43b5-4e66-8b0e-7c3773c23d8e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:35.859346 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1156/2294] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:35 2022] PUT /identity/v3/projects/926c7a8ed8c3437c99cc0afaf36e5ae7/users/4fd8b0ec0f584c6ea0c8b6be042b8f17/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:35.872382 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb00f78-cb61-450e-8d1c-c6335d6b70c9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:35.872672 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb00f78-cb61-450e-8d1c-c6335d6b70c9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:35.872949 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb00f78-cb61-450e-8d1c-c6335d6b70c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:35.902830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-bfb00f78-cb61-450e-8d1c-c6335d6b70c9 None None] MFA Rules not processed for user `4fd8b0ec0f584c6ea0c8b6be042b8f17`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:35.904365 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bfb00f78-cb61-450e-8d1c-c6335d6b70c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:35.986366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1139/2295] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:35.992833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223d26e-43b5-4e66-8b0e-7c3773c23d8e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:36.076496 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223d26e-43b5-4e66-8b0e-7c3773c23d8e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:36.081345 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415ac92-83e4-4a8d-8260-976bd046fc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:36.082009 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415ac92-83e4-4a8d-8260-976bd046fc28 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:36.087352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415ac92-83e4-4a8d-8260-976bd046fc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:36.087352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415ac92-83e4-4a8d-8260-976bd046fc28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:36.087352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415ac92-83e4-4a8d-8260-976bd046fc28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:36.087352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415ac92-83e4-4a8d-8260-976bd046fc28 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:36.170592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1157/2296] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:35 2022] POST /identity/v3/projects => generated 379 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:36.177045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb00f78-cb61-450e-8d1c-c6335d6b70c9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:36.188248 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb00f78-cb61-450e-8d1c-c6335d6b70c9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:36.245577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c12a0-84b2-4d68-af3b-47495749d211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:36.246253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c12a0-84b2-4d68-af3b-47495749d211 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:36.246539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c12a0-84b2-4d68-af3b-47495749d211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:36.246816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c12a0-84b2-4d68-af3b-47495749d211 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:36.248916 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c12a0-84b2-4d68-af3b-47495749d211 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:36.250869 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c12a0-84b2-4d68-af3b-47495749d211 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:36.373508 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1140/2297] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:36 2022] POST /identity/v3/users => generated 290 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:36.377431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415ac92-83e4-4a8d-8260-976bd046fc28 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:36.387764 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415ac92-83e4-4a8d-8260-976bd046fc28 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:36.392234 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27426471-3e7c-4a6e-b4e8-d097ed94d05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:36.392905 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27426471-3e7c-4a6e-b4e8-d097ed94d05a admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:36.393171 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27426471-3e7c-4a6e-b4e8-d097ed94d05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:36.393442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27426471-3e7c-4a6e-b4e8-d097ed94d05a admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:36.394163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27426471-3e7c-4a6e-b4e8-d097ed94d05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:36.395792 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27426471-3e7c-4a6e-b4e8-d097ed94d05a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:36.406531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1158/2298] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:36.411052 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c12a0-84b2-4d68-af3b-47495749d211 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:36.420995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c12a0-84b2-4d68-af3b-47495749d211 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:36.426285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a61c2-fc7a-471d-90f5-6d33eb888f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:36.426938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a61c2-fc7a-471d-90f5-6d33eb888f66 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:36.427257 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a61c2-fc7a-471d-90f5-6d33eb888f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:36.427540 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a61c2-fc7a-471d-90f5-6d33eb888f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:36.428262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a61c2-fc7a-471d-90f5-6d33eb888f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:36.429930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a61c2-fc7a-471d-90f5-6d33eb888f66 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:36.439844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1141/2299] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:36.444019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27426471-3e7c-4a6e-b4e8-d097ed94d05a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:36.453975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27426471-3e7c-4a6e-b4e8-d097ed94d05a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:36.458696 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c564e-6f9a-454d-aeaa-a68ed8e8ee8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:36.459612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c564e-6f9a-454d-aeaa-a68ed8e8ee8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:36.459893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c564e-6f9a-454d-aeaa-a68ed8e8ee8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:36.460167 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c564e-6f9a-454d-aeaa-a68ed8e8ee8b admin admin] PATH_INFO: `/v3/projects/0424cb0f50bf44f5b26110beb3089acc/users/756bc25b0eb8470790d891fe21371293/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:36.474918 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c564e-6f9a-454d-aeaa-a68ed8e8ee8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0424cb0f50bf44f5b26110beb3089acc, user_id=756bc25b0eb8470790d891fe21371293, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:36.477609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c564e-6f9a-454d-aeaa-a68ed8e8ee8b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:36.578933 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1159/2300] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:36 2022] PUT /identity/v3/projects/0424cb0f50bf44f5b26110beb3089acc/users/756bc25b0eb8470790d891fe21371293/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:36.589900 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573537b4-2f5d-4e66-bba8-ef207f1033f7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:36.590227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573537b4-2f5d-4e66-bba8-ef207f1033f7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:36.590464 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573537b4-2f5d-4e66-bba8-ef207f1033f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:36.610582 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:36.619036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-573537b4-2f5d-4e66-bba8-ef207f1033f7 None None] MFA Rules not processed for user `756bc25b0eb8470790d891fe21371293`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:36.620323 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-573537b4-2f5d-4e66-bba8-ef207f1033f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:36.693983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1142/2301] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:36.698486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c564e-6f9a-454d-aeaa-a68ed8e8ee8b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:36.763952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c564e-6f9a-454d-aeaa-a68ed8e8ee8b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:36.778861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d1174-939d-44a2-9ced-9a298db1746c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:36.778861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d1174-939d-44a2-9ced-9a298db1746c admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:36.778861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d1174-939d-44a2-9ced-9a298db1746c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:36.778861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d1174-939d-44a2-9ced-9a298db1746c admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:36.779529 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d1174-939d-44a2-9ced-9a298db1746c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:36.779529 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d1174-939d-44a2-9ced-9a298db1746c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:36.868094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1160/2302] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:36 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:36.872351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573537b4-2f5d-4e66-bba8-ef207f1033f7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:36.884152 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573537b4-2f5d-4e66-bba8-ef207f1033f7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:36.977285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dab832-f2a3-421a-87b7-d37e664f2918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:36.981802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dab832-f2a3-421a-87b7-d37e664f2918 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:36.982195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dab832-f2a3-421a-87b7-d37e664f2918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:36.982615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dab832-f2a3-421a-87b7-d37e664f2918 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:36.983475 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dab832-f2a3-421a-87b7-d37e664f2918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:36.985785 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dab832-f2a3-421a-87b7-d37e664f2918 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:37.109049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1143/2303] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:36 2022] POST /identity/v3/users => generated 294 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:37.112598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d1174-939d-44a2-9ced-9a298db1746c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:37.129147 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d1174-939d-44a2-9ced-9a298db1746c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:37.138876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26375f-d303-4c5f-ba04-cc68fc495561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:37.139668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26375f-d303-4c5f-ba04-cc68fc495561 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:37.139957 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26375f-d303-4c5f-ba04-cc68fc495561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:37.140238 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26375f-d303-4c5f-ba04-cc68fc495561 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:37.141030 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26375f-d303-4c5f-ba04-cc68fc495561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:37.143351 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26375f-d303-4c5f-ba04-cc68fc495561 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:37.180078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1161/2304] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:37.183566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dab832-f2a3-421a-87b7-d37e664f2918 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:37.195051 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dab832-f2a3-421a-87b7-d37e664f2918 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:37.200597 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a15dc-9770-4a6c-825b-2f58dc6953ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:37.202501 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a15dc-9770-4a6c-825b-2f58dc6953ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:37.202501 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a15dc-9770-4a6c-825b-2f58dc6953ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:37.202501 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a15dc-9770-4a6c-825b-2f58dc6953ca admin admin] PATH_INFO: `/v3/projects/2075d37157814da2bc43dafdd72d1f9d/users/b0433aa9c1fd468e8cebe6c55e9866e8/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:37.220607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a15dc-9770-4a6c-825b-2f58dc6953ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=2075d37157814da2bc43dafdd72d1f9d, user_id=b0433aa9c1fd468e8cebe6c55e9866e8, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:37.223321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a15dc-9770-4a6c-825b-2f58dc6953ca admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:37.368189 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1144/2305] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:37 2022] PUT /identity/v3/projects/2075d37157814da2bc43dafdd72d1f9d/users/b0433aa9c1fd468e8cebe6c55e9866e8/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:37.372191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26375f-d303-4c5f-ba04-cc68fc495561 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:37.383068 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26375f-d303-4c5f-ba04-cc68fc495561 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:37.394761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:37.444613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76bb3e-c4e4-4355-a2a3-5c27c0efaec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:37.445293 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76bb3e-c4e4-4355-a2a3-5c27c0efaec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:37.445581 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76bb3e-c4e4-4355-a2a3-5c27c0efaec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:37.445856 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76bb3e-c4e4-4355-a2a3-5c27c0efaec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:37.446803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76bb3e-c4e4-4355-a2a3-5c27c0efaec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:37.448445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76bb3e-c4e4-4355-a2a3-5c27c0efaec5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:37.459021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1162/2306] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:37.462983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a15dc-9770-4a6c-825b-2f58dc6953ca admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:37.473825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a15dc-9770-4a6c-825b-2f58dc6953ca admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:37.479070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c358f-1b7b-446d-9b84-df8519978a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:37.480337 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c358f-1b7b-446d-9b84-df8519978a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:37.480696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c358f-1b7b-446d-9b84-df8519978a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:37.481052 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c358f-1b7b-446d-9b84-df8519978a92 admin admin] PATH_INFO: `/v3/projects/2075d37157814da2bc43dafdd72d1f9d/users/b0433aa9c1fd468e8cebe6c55e9866e8/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:37.483076 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c358f-1b7b-446d-9b84-df8519978a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2075d37157814da2bc43dafdd72d1f9d, user_id=b0433aa9c1fd468e8cebe6c55e9866e8, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:37.486343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c358f-1b7b-446d-9b84-df8519978a92 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:37.497699 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1145/2307] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:37 2022] PUT /identity/v3/projects/2075d37157814da2bc43dafdd72d1f9d/users/b0433aa9c1fd468e8cebe6c55e9866e8/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:37.509231 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c528bc4-7b90-421d-8378-556fddea8311 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:37.509519 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c528bc4-7b90-421d-8378-556fddea8311 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:37.509807 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c528bc4-7b90-421d-8378-556fddea8311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:37.536119 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8c528bc4-7b90-421d-8378-556fddea8311 None None] MFA Rules not processed for user `b0433aa9c1fd468e8cebe6c55e9866e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:37.537490 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8c528bc4-7b90-421d-8378-556fddea8311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:37.605682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1163/2308] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:37 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:37.617667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c358f-1b7b-446d-9b84-df8519978a92 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:37.624024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-927c358f-1b7b-446d-9b84-df8519978a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:37.684808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c358f-1b7b-446d-9b84-df8519978a92 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:37.688722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884e16b-c45a-4cff-bdb2-5752dddbd4d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:37.690174 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884e16b-c45a-4cff-bdb2-5752dddbd4d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:37.690174 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884e16b-c45a-4cff-bdb2-5752dddbd4d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:37.690174 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884e16b-c45a-4cff-bdb2-5752dddbd4d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:37.696766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:37.700210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884e16b-c45a-4cff-bdb2-5752dddbd4d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:37.701714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884e16b-c45a-4cff-bdb2-5752dddbd4d9 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:37.712204 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1146/2309] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:37.801818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c528bc4-7b90-421d-8378-556fddea8311 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:37.811975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c528bc4-7b90-421d-8378-556fddea8311 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:37.815858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07214d1-fd84-48a3-b432-382231ee4279 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:37.816499 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07214d1-fd84-48a3-b432-382231ee4279 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:37.816780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07214d1-fd84-48a3-b432-382231ee4279 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:37.817065 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07214d1-fd84-48a3-b432-382231ee4279 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:37.880009 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07214d1-fd84-48a3-b432-382231ee4279 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:37.880009 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07214d1-fd84-48a3-b432-382231ee4279 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:37.888547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1164/2310] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:41.476347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884e16b-c45a-4cff-bdb2-5752dddbd4d9 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:41.478604 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3884e16b-c45a-4cff-bdb2-5752dddbd4d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:41.547133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884e16b-c45a-4cff-bdb2-5752dddbd4d9 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:41.551999 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c5a5f-d726-4625-8a14-7304a1be9e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:41.552875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c5a5f-d726-4625-8a14-7304a1be9e86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:41.553159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c5a5f-d726-4625-8a14-7304a1be9e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:41.553438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c5a5f-d726-4625-8a14-7304a1be9e86 admin admin] PATH_INFO: `/v3/users/be6528b7fcfc4cdd914d3d86ef86dff9` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:41.554623 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c5a5f-d726-4625-8a14-7304a1be9e86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be6528b7fcfc4cdd914d3d86ef86dff9)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:41.556577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c5a5f-d726-4625-8a14-7304a1be9e86 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:41.730178 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-543c5a5f-d726-4625-8a14-7304a1be9e86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be6528b7fcfc4cdd914d3d86ef86dff9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:41.743123 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-543c5a5f-d726-4625-8a14-7304a1be9e86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be6528b7fcfc4cdd914d3d86ef86dff9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:41.745648 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-543c5a5f-d726-4625-8a14-7304a1be9e86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be6528b7fcfc4cdd914d3d86ef86dff9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:41.751589 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-543c5a5f-d726-4625-8a14-7304a1be9e86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be6528b7fcfc4cdd914d3d86ef86dff9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:41.752237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-543c5a5f-d726-4625-8a14-7304a1be9e86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be6528b7fcfc4cdd914d3d86ef86dff9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:41.852430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1147/2311] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:41 2022] DELETE /identity/v3/users/be6528b7fcfc4cdd914d3d86ef86dff9 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:41.862325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79904c-7475-41ea-9e4e-58984ba5d162 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:41.863430 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79904c-7475-41ea-9e4e-58984ba5d162 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:41.863659 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79904c-7475-41ea-9e4e-58984ba5d162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:41.882628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fc79904c-7475-41ea-9e4e-58984ba5d162 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:41.892098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fc79904c-7475-41ea-9e4e-58984ba5d162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:41.920427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:41.951410 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1165/2312] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:41.968090 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c5a5f-d726-4625-8a14-7304a1be9e86 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:41.968090 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-543c5a5f-d726-4625-8a14-7304a1be9e86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:42.014864 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79904c-7475-41ea-9e4e-58984ba5d162 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:42.016434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fc79904c-7475-41ea-9e4e-58984ba5d162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:42.031297 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79904c-7475-41ea-9e4e-58984ba5d162 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:42.034259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c5a5f-d726-4625-8a14-7304a1be9e86 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:42.036838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c83d1-ab2f-4db2-92dc-9db6122a4839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:42.037731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c83d1-ab2f-4db2-92dc-9db6122a4839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:42.038134 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c83d1-ab2f-4db2-92dc-9db6122a4839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:42.038408 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c83d1-ab2f-4db2-92dc-9db6122a4839 admin admin] PATH_INFO: `/v3/users/905186a697e44d77b94fc2bb20302b12` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:42.039646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c83d1-ab2f-4db2-92dc-9db6122a4839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=905186a697e44d77b94fc2bb20302b12)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:42.041282 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2697e75-ca04-45a9-ab23-c923d8cb6009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:42.042023 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c83d1-ab2f-4db2-92dc-9db6122a4839 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:42.043674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2697e75-ca04-45a9-ab23-c923d8cb6009 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:42.043908 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2697e75-ca04-45a9-ab23-c923d8cb6009 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:42.044184 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2697e75-ca04-45a9-ab23-c923d8cb6009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:42.065558 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2697e75-ca04-45a9-ab23-c923d8cb6009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:42.067846 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2697e75-ca04-45a9-ab23-c923d8cb6009 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:42.079307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1148/2313] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:20:42.210274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2697e75-ca04-45a9-ab23-c923d8cb6009 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:42.211749 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c2697e75-ca04-45a9-ab23-c923d8cb6009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:42.226123 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2697e75-ca04-45a9-ab23-c923d8cb6009 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:42.230646 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12019e2-4076-4e59-bcab-aa0bdb71e113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:42.231536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12019e2-4076-4e59-bcab-aa0bdb71e113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:42.231820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12019e2-4076-4e59-bcab-aa0bdb71e113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:42.232093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12019e2-4076-4e59-bcab-aa0bdb71e113 admin admin] PATH_INFO: `/v3/users/4fd8b0ec0f584c6ea0c8b6be042b8f17` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:42.233264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12019e2-4076-4e59-bcab-aa0bdb71e113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fd8b0ec0f584c6ea0c8b6be042b8f17)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:42.235150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12019e2-4076-4e59-bcab-aa0bdb71e113 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:42.246481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:42.286458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-320c83d1-ab2f-4db2-92dc-9db6122a4839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '905186a697e44d77b94fc2bb20302b12'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:42.303860 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-320c83d1-ab2f-4db2-92dc-9db6122a4839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '905186a697e44d77b94fc2bb20302b12'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:42.429896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-320c83d1-ab2f-4db2-92dc-9db6122a4839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '905186a697e44d77b94fc2bb20302b12'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:42.430533 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-320c83d1-ab2f-4db2-92dc-9db6122a4839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '905186a697e44d77b94fc2bb20302b12'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:42.438160 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-320c83d1-ab2f-4db2-92dc-9db6122a4839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '905186a697e44d77b94fc2bb20302b12'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:42.444960 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1166/2314] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:42 2022] DELETE /identity/v3/users/905186a697e44d77b94fc2bb20302b12 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:42.448948 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4faaf-481f-4cd5-9ca7-6f4f924eadbf None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:42.449235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4faaf-481f-4cd5-9ca7-6f4f924eadbf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:42.449506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4faaf-481f-4cd5-9ca7-6f4f924eadbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:42.466762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-67c4faaf-481f-4cd5-9ca7-6f4f924eadbf None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:42.467984 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-67c4faaf-481f-4cd5-9ca7-6f4f924eadbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:42.508998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a12019e2-4076-4e59-bcab-aa0bdb71e113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fd8b0ec0f584c6ea0c8b6be042b8f17'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:42.532720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a12019e2-4076-4e59-bcab-aa0bdb71e113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fd8b0ec0f584c6ea0c8b6be042b8f17'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:42.532720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a12019e2-4076-4e59-bcab-aa0bdb71e113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fd8b0ec0f584c6ea0c8b6be042b8f17'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:42.540543 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1167/2315] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:42.541799 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4faaf-481f-4cd5-9ca7-6f4f924eadbf None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:42.542310 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a12019e2-4076-4e59-bcab-aa0bdb71e113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fd8b0ec0f584c6ea0c8b6be042b8f17'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:42.543377 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-67c4faaf-481f-4cd5-9ca7-6f4f924eadbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:42.543707 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a12019e2-4076-4e59-bcab-aa0bdb71e113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fd8b0ec0f584c6ea0c8b6be042b8f17'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:42.557144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4faaf-481f-4cd5-9ca7-6f4f924eadbf None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:42.562660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd781aa-f961-4ddc-9a66-55a0bd8ce36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:42.569357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd781aa-f961-4ddc-9a66-55a0bd8ce36c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:42.569357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd781aa-f961-4ddc-9a66-55a0bd8ce36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:42.569357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd781aa-f961-4ddc-9a66-55a0bd8ce36c admin admin] PATH_INFO: `/v3/projects/4a4a374fe5f64ff6ba0b37f62d2d5f4d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:42.569357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd781aa-f961-4ddc-9a66-55a0bd8ce36c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a4a374fe5f64ff6ba0b37f62d2d5f4d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:42.570557 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd781aa-f961-4ddc-9a66-55a0bd8ce36c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:42.618581 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1149/2316] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:42 2022] DELETE /identity/v3/users/4fd8b0ec0f584c6ea0c8b6be042b8f17 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:42.619599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12019e2-4076-4e59-bcab-aa0bdb71e113 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:42.620962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a12019e2-4076-4e59-bcab-aa0bdb71e113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:42.678934 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12019e2-4076-4e59-bcab-aa0bdb71e113 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:42.680694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:42.684373 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4cd781aa-f961-4ddc-9a66-55a0bd8ce36c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a4a374fe5f64ff6ba0b37f62d2d5f4d', 'request_id': 'req-4cd781aa-f961-4ddc-9a66-55a0bd8ce36c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:42.690540 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4cd781aa-f961-4ddc-9a66-55a0bd8ce36c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a4a374fe5f64ff6ba0b37f62d2d5f4d', 'request_id': 'req-4cd781aa-f961-4ddc-9a66-55a0bd8ce36c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:42.731290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a136c-4b85-45a1-9742-8ef4f3ea3b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:42.732727 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a136c-4b85-45a1-9742-8ef4f3ea3b0a service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:42.732727 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a136c-4b85-45a1-9742-8ef4f3ea3b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:42.732727 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a136c-4b85-45a1-9742-8ef4f3ea3b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:42.734332 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-744a136c-4b85-45a1-9742-8ef4f3ea3b0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:42.751701 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4cd781aa-f961-4ddc-9a66-55a0bd8ce36c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a4a374fe5f64ff6ba0b37f62d2d5f4d', 'request_id': 'req-4cd781aa-f961-4ddc-9a66-55a0bd8ce36c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:42.764413 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4cd781aa-f961-4ddc-9a66-55a0bd8ce36c admin admin] The token cache is being invalidate because project 4a4a374fe5f64ff6ba0b37f62d2d5f4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:42.764765 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4cd781aa-f961-4ddc-9a66-55a0bd8ce36c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:42.765788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4cd781aa-f961-4ddc-9a66-55a0bd8ce36c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:42.768492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1168/2317] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:42 2022] DELETE /identity/v3/projects/4a4a374fe5f64ff6ba0b37f62d2d5f4d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:42.769871 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a8423-034e-4b2d-b243-81990a8b93a5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:42.770322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a8423-034e-4b2d-b243-81990a8b93a5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:42.770670 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a8423-034e-4b2d-b243-81990a8b93a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:42.790259 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-851a8423-034e-4b2d-b243-81990a8b93a5 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:42.791660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-851a8423-034e-4b2d-b243-81990a8b93a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:42.808543 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:42.813290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a136c-4b85-45a1-9742-8ef4f3ea3b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:42.814783 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a136c-4b85-45a1-9742-8ef4f3ea3b0a service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:42.826235 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1150/2318] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:42.829953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b135ed0-a8df-49d8-bb5d-1ba10db326a4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:42.829953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b135ed0-a8df-49d8-bb5d-1ba10db326a4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:42.829953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b135ed0-a8df-49d8-bb5d-1ba10db326a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:42.846556 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8b135ed0-a8df-49d8-bb5d-1ba10db326a4 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:42.848571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8b135ed0-a8df-49d8-bb5d-1ba10db326a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:42.863977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1151/2319] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:42.864103 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1169/2320] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:42.865642 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e67b4-e5fe-4adf-a2bc-0bc47ad4afec None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:42.866072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e67b4-e5fe-4adf-a2bc-0bc47ad4afec None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:42.866452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e67b4-e5fe-4adf-a2bc-0bc47ad4afec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:42.868577 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a8423-034e-4b2d-b243-81990a8b93a5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:42.871207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-851a8423-034e-4b2d-b243-81990a8b93a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:42.895335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a91e67b4-e5fe-4adf-a2bc-0bc47ad4afec None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:42.895446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a8423-034e-4b2d-b243-81990a8b93a5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:42.895539 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a91e67b4-e5fe-4adf-a2bc-0bc47ad4afec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:42.895634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2636025-e768-4880-befb-84e002e30195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:42.895634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2636025-e768-4880-befb-84e002e30195 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:42.895634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2636025-e768-4880-befb-84e002e30195 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:42.895634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2636025-e768-4880-befb-84e002e30195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:42.900923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1152/2321] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:42.905491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e67b4-e5fe-4adf-a2bc-0bc47ad4afec None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:42.905981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2636025-e768-4880-befb-84e002e30195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:42.907676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2636025-e768-4880-befb-84e002e30195 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:42.916451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e67b4-e5fe-4adf-a2bc-0bc47ad4afec None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:42.918779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1170/2322] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:42.921823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4a1b3-9534-44d0-9afb-19679af6c810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:42.922572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4a1b3-9534-44d0-9afb-19679af6c810 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:42.922979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4a1b3-9534-44d0-9afb-19679af6c810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:42.923366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4a1b3-9534-44d0-9afb-19679af6c810 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:42.924199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4a1b3-9534-44d0-9afb-19679af6c810 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:42.924577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4a1b3-9534-44d0-9afb-19679af6c810 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:42.927305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4a1b3-9534-44d0-9afb-19679af6c810 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:42.940019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1153/2323] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:20:42.944607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2636025-e768-4880-befb-84e002e30195 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:42.954831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2636025-e768-4880-befb-84e002e30195 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:42.959913 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a1eeb-6194-4e17-acda-b5526fc62a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:42.960515 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a1eeb-6194-4e17-acda-b5526fc62a36 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:42.960805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a1eeb-6194-4e17-acda-b5526fc62a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:42.961118 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a1eeb-6194-4e17-acda-b5526fc62a36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:42.961982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a1eeb-6194-4e17-acda-b5526fc62a36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:42.963989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a1eeb-6194-4e17-acda-b5526fc62a36 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:43.066541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1171/2324] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:42 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:43.070577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4a1b3-9534-44d0-9afb-19679af6c810 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:43.080794 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4a1b3-9534-44d0-9afb-19679af6c810 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:43.143906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ad2bc-ad84-4cf6-908d-6ae1d1438505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:43.144606 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ad2bc-ad84-4cf6-908d-6ae1d1438505 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:43.144898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ad2bc-ad84-4cf6-908d-6ae1d1438505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:43.145172 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ad2bc-ad84-4cf6-908d-6ae1d1438505 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:43.145863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ad2bc-ad84-4cf6-908d-6ae1d1438505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:43.147817 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ad2bc-ad84-4cf6-908d-6ae1d1438505 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:43.256043 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1154/2325] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:43 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:43.257320 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a1eeb-6194-4e17-acda-b5526fc62a36 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:43.270050 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a1eeb-6194-4e17-acda-b5526fc62a36 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:43.273671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d717b1-f8bb-4159-9073-3c8aae73b160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:43.274340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d717b1-f8bb-4159-9073-3c8aae73b160 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:43.274628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d717b1-f8bb-4159-9073-3c8aae73b160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:43.274908 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d717b1-f8bb-4159-9073-3c8aae73b160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:43.275845 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d717b1-f8bb-4159-9073-3c8aae73b160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:43.277430 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d717b1-f8bb-4159-9073-3c8aae73b160 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:43.289824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1172/2326] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:43.294104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ad2bc-ad84-4cf6-908d-6ae1d1438505 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:43.304402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ad2bc-ad84-4cf6-908d-6ae1d1438505 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:43.309209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1d041-2fdb-4903-91f9-0a64b66c30a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:43.310981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1d041-2fdb-4903-91f9-0a64b66c30a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:43.311755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1d041-2fdb-4903-91f9-0a64b66c30a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:43.311755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1d041-2fdb-4903-91f9-0a64b66c30a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:43.312521 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1d041-2fdb-4903-91f9-0a64b66c30a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:43.314256 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1d041-2fdb-4903-91f9-0a64b66c30a0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:43.324787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1155/2327] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:43.329004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d717b1-f8bb-4159-9073-3c8aae73b160 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:43.339859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d717b1-f8bb-4159-9073-3c8aae73b160 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:43.345094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09a1b1-9840-47bd-b46d-3c989a01460b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:43.352787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09a1b1-9840-47bd-b46d-3c989a01460b admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:43.352787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09a1b1-9840-47bd-b46d-3c989a01460b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:43.352787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09a1b1-9840-47bd-b46d-3c989a01460b admin admin] PATH_INFO: `/v3/projects/176337a3913047eb8d63d085b8a5db45/users/4f828fa6c46c4816945ef2cd77a33e8c/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:43.354021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1d041-2fdb-4903-91f9-0a64b66c30a0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:43.355954 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-32d1d041-2fdb-4903-91f9-0a64b66c30a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:43.368087 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09a1b1-9840-47bd-b46d-3c989a01460b admin admin] RBAC: Authorizing `identity:create_grant(project_id=176337a3913047eb8d63d085b8a5db45, user_id=4f828fa6c46c4816945ef2cd77a33e8c, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:43.370804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09a1b1-9840-47bd-b46d-3c989a01460b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:43.373493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1d041-2fdb-4903-91f9-0a64b66c30a0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:43.378343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300c710-ee52-439e-a386-4d829c08d20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:43.379259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300c710-ee52-439e-a386-4d829c08d20d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:43.379543 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300c710-ee52-439e-a386-4d829c08d20d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:43.379815 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300c710-ee52-439e-a386-4d829c08d20d admin admin] PATH_INFO: `/v3/projects/aec47099c2e74ba3a1d5cf86fb6456f5` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:43.381053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300c710-ee52-439e-a386-4d829c08d20d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aec47099c2e74ba3a1d5cf86fb6456f5)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:43.382903 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300c710-ee52-439e-a386-4d829c08d20d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:43.469164 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1173/2328] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:43 2022] PUT /identity/v3/projects/176337a3913047eb8d63d085b8a5db45/users/4f828fa6c46c4816945ef2cd77a33e8c/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:43.474950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09a1b1-9840-47bd-b46d-3c989a01460b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:43.477726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4f09a1b1-9840-47bd-b46d-3c989a01460b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:43.493163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3300c710-ee52-439e-a386-4d829c08d20d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aec47099c2e74ba3a1d5cf86fb6456f5', 'request_id': 'req-3300c710-ee52-439e-a386-4d829c08d20d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:43.500049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3300c710-ee52-439e-a386-4d829c08d20d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aec47099c2e74ba3a1d5cf86fb6456f5', 'request_id': 'req-3300c710-ee52-439e-a386-4d829c08d20d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:43.546856 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09a1b1-9840-47bd-b46d-3c989a01460b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:43.551420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:43.574380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3300c710-ee52-439e-a386-4d829c08d20d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aec47099c2e74ba3a1d5cf86fb6456f5', 'request_id': 'req-3300c710-ee52-439e-a386-4d829c08d20d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:43.590093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3300c710-ee52-439e-a386-4d829c08d20d admin admin] The token cache is being invalidate because project aec47099c2e74ba3a1d5cf86fb6456f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:43.590311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3300c710-ee52-439e-a386-4d829c08d20d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:43.590973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3300c710-ee52-439e-a386-4d829c08d20d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:43.592707 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1156/2329] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:43 2022] DELETE /identity/v3/projects/aec47099c2e74ba3a1d5cf86fb6456f5 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:43.595725 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c9e08-0129-4461-9414-310271bc48b6 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:43.595725 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c9e08-0129-4461-9414-310271bc48b6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:43.595725 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c9e08-0129-4461-9414-310271bc48b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:43.620005 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc620c-1072-4262-bb7a-3817009c807f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:43.621454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc620c-1072-4262-bb7a-3817009c807f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:43.621923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc620c-1072-4262-bb7a-3817009c807f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:43.622393 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc620c-1072-4262-bb7a-3817009c807f admin admin] PATH_INFO: `/v3/projects/926c7a8ed8c3437c99cc0afaf36e5ae7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:43.624369 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc620c-1072-4262-bb7a-3817009c807f admin admin] RBAC: Authorizing `identity:delete_project(project_id=926c7a8ed8c3437c99cc0afaf36e5ae7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:43.627895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc620c-1072-4262-bb7a-3817009c807f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:43.631199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-647c9e08-0129-4461-9414-310271bc48b6 None None] MFA Rules not processed for user `4f828fa6c46c4816945ef2cd77a33e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:43.633530 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-647c9e08-0129-4461-9414-310271bc48b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:43.666079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:43.715059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1157/2330] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:43.715944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c9e08-0129-4461-9414-310271bc48b6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:43.717213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-647c9e08-0129-4461-9414-310271bc48b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:43.730787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c9e08-0129-4461-9414-310271bc48b6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:43.736464 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb4f54-8949-417d-b473-725069f28342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:43.737517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb4f54-8949-417d-b473-725069f28342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:43.738081 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb4f54-8949-417d-b473-725069f28342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:43.738695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb4f54-8949-417d-b473-725069f28342 admin admin] PATH_INFO: `/v3/users/2f2949e42d2f4c4189912f55ea4bfc5e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:43.740061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb4f54-8949-417d-b473-725069f28342 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f2949e42d2f4c4189912f55ea4bfc5e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:43.742067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb4f54-8949-417d-b473-725069f28342 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:43.748777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-94dc620c-1072-4262-bb7a-3817009c807f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '926c7a8ed8c3437c99cc0afaf36e5ae7', 'request_id': 'req-94dc620c-1072-4262-bb7a-3817009c807f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:43.755903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-94dc620c-1072-4262-bb7a-3817009c807f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '926c7a8ed8c3437c99cc0afaf36e5ae7', 'request_id': 'req-94dc620c-1072-4262-bb7a-3817009c807f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:43.826075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-94dc620c-1072-4262-bb7a-3817009c807f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '926c7a8ed8c3437c99cc0afaf36e5ae7', 'request_id': 'req-94dc620c-1072-4262-bb7a-3817009c807f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:43.839832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-94dc620c-1072-4262-bb7a-3817009c807f admin admin] The token cache is being invalidate because project 926c7a8ed8c3437c99cc0afaf36e5ae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:43.840125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-94dc620c-1072-4262-bb7a-3817009c807f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:43.840785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-94dc620c-1072-4262-bb7a-3817009c807f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:43.842352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1174/2331] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:43 2022] DELETE /identity/v3/projects/926c7a8ed8c3437c99cc0afaf36e5ae7 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:43.843304 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc620c-1072-4262-bb7a-3817009c807f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:43.844577 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-94dc620c-1072-4262-bb7a-3817009c807f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:43.902325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc620c-1072-4262-bb7a-3817009c807f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:43.906177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2dadd-d625-4a2f-81ed-175ec5e6efde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:43.906838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2dadd-d625-4a2f-81ed-175ec5e6efde service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:43.907127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2dadd-d625-4a2f-81ed-175ec5e6efde service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:43.907432 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2dadd-d625-4a2f-81ed-175ec5e6efde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:43.909349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-83d2dadd-d625-4a2f-81ed-175ec5e6efde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:43.961733 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a8fb4f54-8949-417d-b473-725069f28342 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f2949e42d2f4c4189912f55ea4bfc5e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:43.972529 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2dadd-d625-4a2f-81ed-175ec5e6efde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:43.973510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a8fb4f54-8949-417d-b473-725069f28342 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f2949e42d2f4c4189912f55ea4bfc5e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:43.974205 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a8fb4f54-8949-417d-b473-725069f28342 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f2949e42d2f4c4189912f55ea4bfc5e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:43.974318 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2dadd-d625-4a2f-81ed-175ec5e6efde service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:43.984224 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1175/2332] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:43.989323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2dadd-d625-4a2f-81ed-175ec5e6efde service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:43.989323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-83d2dadd-d625-4a2f-81ed-175ec5e6efde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:43.995742 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a8fb4f54-8949-417d-b473-725069f28342 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f2949e42d2f4c4189912f55ea4bfc5e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:43.996706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a8fb4f54-8949-417d-b473-725069f28342 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f2949e42d2f4c4189912f55ea4bfc5e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:44.063341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2dadd-d625-4a2f-81ed-175ec5e6efde service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:44.067919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:44.074162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c43f47-669c-4a74-a6b6-b7dde4027506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:44.074697 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1158/2333] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:43 2022] DELETE /identity/v3/users/2f2949e42d2f4c4189912f55ea4bfc5e => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:44.078427 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c43f47-669c-4a74-a6b6-b7dde4027506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:44.078657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c43f47-669c-4a74-a6b6-b7dde4027506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:44.079007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c43f47-669c-4a74-a6b6-b7dde4027506 admin admin] PATH_INFO: `/v3/users/756bc25b0eb8470790d891fe21371293` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:44.080368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c43f47-669c-4a74-a6b6-b7dde4027506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=756bc25b0eb8470790d891fe21371293)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:44.082724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c43f47-669c-4a74-a6b6-b7dde4027506 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:44.319367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-10c43f47-669c-4a74-a6b6-b7dde4027506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '756bc25b0eb8470790d891fe21371293'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:44.332325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-10c43f47-669c-4a74-a6b6-b7dde4027506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '756bc25b0eb8470790d891fe21371293'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:44.425610 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-10c43f47-669c-4a74-a6b6-b7dde4027506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '756bc25b0eb8470790d891fe21371293'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:44.426859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-10c43f47-669c-4a74-a6b6-b7dde4027506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '756bc25b0eb8470790d891fe21371293'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:44.434766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-10c43f47-669c-4a74-a6b6-b7dde4027506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '756bc25b0eb8470790d891fe21371293'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:44.445594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1176/2334] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:43 2022] DELETE /identity/v3/users/756bc25b0eb8470790d891fe21371293 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:44.553816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb4f54-8949-417d-b473-725069f28342 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:44.555524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a8fb4f54-8949-417d-b473-725069f28342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:44.677995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb4f54-8949-417d-b473-725069f28342 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:44.736848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:44.766828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:44.768112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:44.768402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:44.769330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a admin admin] PATH_INFO: `/v3/projects/ada77210f6f04fb1bf376da287ff7236` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:44.771564 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ada77210f6f04fb1bf376da287ff7236)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:44.774307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:44.939313 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ada77210f6f04fb1bf376da287ff7236', 'request_id': 'req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:44.947631 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ada77210f6f04fb1bf376da287ff7236', 'request_id': 'req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:45.026622 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ada77210f6f04fb1bf376da287ff7236', 'request_id': 'req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:45.038588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a admin admin] The token cache is being invalidate because project ada77210f6f04fb1bf376da287ff7236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:45.038874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:45.039584 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:45.041128 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1159/2335] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:44 2022] DELETE /identity/v3/projects/ada77210f6f04fb1bf376da287ff7236 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:45.046255 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abe010-c89e-49f1-bd38-a3e803fc787d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:45.046727 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abe010-c89e-49f1-bd38-a3e803fc787d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:45.046820 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abe010-c89e-49f1-bd38-a3e803fc787d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:45.064941 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-25abe010-c89e-49f1-bd38-a3e803fc787d None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:45.066916 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-25abe010-c89e-49f1-bd38-a3e803fc787d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:45.104745 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:45.106433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:45.165219 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1177/2336] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:45.172899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdc53c-f0f8-4045-8498-c59dd3338d46 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:45.173257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdc53c-f0f8-4045-8498-c59dd3338d46 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:45.173588 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdc53c-f0f8-4045-8498-c59dd3338d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:45.193340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c0fdc53c-f0f8-4045-8498-c59dd3338d46 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:45.194640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c0fdc53c-f0f8-4045-8498-c59dd3338d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:45.210231 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1178/2337] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:45.213246 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85b818-de4c-4fae-a6b7-c5bf22c7173a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:45.214337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fdc53c-f0f8-4045-8498-c59dd3338d46 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:45.219811 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67b41e-5772-4675-99f4-3a23cd3cfee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:45.220913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67b41e-5772-4675-99f4-3a23cd3cfee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:45.221389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67b41e-5772-4675-99f4-3a23cd3cfee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:45.222688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67b41e-5772-4675-99f4-3a23cd3cfee9 admin admin] PATH_INFO: `/v3/projects/0424cb0f50bf44f5b26110beb3089acc` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:45.224380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67b41e-5772-4675-99f4-3a23cd3cfee9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0424cb0f50bf44f5b26110beb3089acc)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:45.227863 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fdc53c-f0f8-4045-8498-c59dd3338d46 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:45.228551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67b41e-5772-4675-99f4-3a23cd3cfee9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:45.231199 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8424d11-bb18-4079-8bda-d1e102825fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:45.231963 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8424d11-bb18-4079-8bda-d1e102825fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:45.232316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8424d11-bb18-4079-8bda-d1e102825fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:45.232777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8424d11-bb18-4079-8bda-d1e102825fa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:45.233583 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8424d11-bb18-4079-8bda-d1e102825fa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:45.233970 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8424d11-bb18-4079-8bda-d1e102825fa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:45.236332 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8424d11-bb18-4079-8bda-d1e102825fa6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:45.239215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:45.254563 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1179/2338] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:20:45.259608 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8424d11-bb18-4079-8bda-d1e102825fa6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:45.270675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8424d11-bb18-4079-8bda-d1e102825fa6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:45.276589 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46944b13-8816-42a6-a106-80ebc8e38eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:45.276832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46944b13-8816-42a6-a106-80ebc8e38eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:45.277269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46944b13-8816-42a6-a106-80ebc8e38eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:45.277557 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46944b13-8816-42a6-a106-80ebc8e38eb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:45.278261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46944b13-8816-42a6-a106-80ebc8e38eb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:45.280502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46944b13-8816-42a6-a106-80ebc8e38eb0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:45.384538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1a67b41e-5772-4675-99f4-3a23cd3cfee9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0424cb0f50bf44f5b26110beb3089acc', 'request_id': 'req-1a67b41e-5772-4675-99f4-3a23cd3cfee9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:45.390987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1a67b41e-5772-4675-99f4-3a23cd3cfee9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0424cb0f50bf44f5b26110beb3089acc', 'request_id': 'req-1a67b41e-5772-4675-99f4-3a23cd3cfee9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:45.410941 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1180/2339] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:45 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:45.415561 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46944b13-8816-42a6-a106-80ebc8e38eb0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:45.469701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1a67b41e-5772-4675-99f4-3a23cd3cfee9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0424cb0f50bf44f5b26110beb3089acc', 'request_id': 'req-1a67b41e-5772-4675-99f4-3a23cd3cfee9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:45.476398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46944b13-8816-42a6-a106-80ebc8e38eb0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:45.481281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875030a7-45d8-4d00-8daf-24f2de1eac3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:45.481953 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875030a7-45d8-4d00-8daf-24f2de1eac3c admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:45.482149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1a67b41e-5772-4675-99f4-3a23cd3cfee9 admin admin] The token cache is being invalidate because project 0424cb0f50bf44f5b26110beb3089acc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:45.482252 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875030a7-45d8-4d00-8daf-24f2de1eac3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:45.482500 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875030a7-45d8-4d00-8daf-24f2de1eac3c admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:45.482998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1a67b41e-5772-4675-99f4-3a23cd3cfee9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:45.483219 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875030a7-45d8-4d00-8daf-24f2de1eac3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:45.483731 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1a67b41e-5772-4675-99f4-3a23cd3cfee9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:45.485214 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875030a7-45d8-4d00-8daf-24f2de1eac3c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:45.485477 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1160/2340] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:45 2022] DELETE /identity/v3/projects/0424cb0f50bf44f5b26110beb3089acc => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:45.490336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67b41e-5772-4675-99f4-3a23cd3cfee9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:45.491848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1a67b41e-5772-4675-99f4-3a23cd3cfee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:45.561550 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67b41e-5772-4675-99f4-3a23cd3cfee9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:45.566891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f56fdf-e69a-4348-95f2-3c7badc12c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:45.567811 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f56fdf-e69a-4348-95f2-3c7badc12c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:45.568099 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f56fdf-e69a-4348-95f2-3c7badc12c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:45.568407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f56fdf-e69a-4348-95f2-3c7badc12c43 admin admin] PATH_INFO: `/v3/users/b0433aa9c1fd468e8cebe6c55e9866e8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:45.569611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f56fdf-e69a-4348-95f2-3c7badc12c43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0433aa9c1fd468e8cebe6c55e9866e8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:45.571568 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f56fdf-e69a-4348-95f2-3c7badc12c43 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:45.618552 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1181/2341] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:45 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:45.619732 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875030a7-45d8-4d00-8daf-24f2de1eac3c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:45.621405 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-875030a7-45d8-4d00-8daf-24f2de1eac3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:45.634275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875030a7-45d8-4d00-8daf-24f2de1eac3c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:45.639106 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94df83-537f-49be-acdd-2d2156463bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:45.639741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94df83-537f-49be-acdd-2d2156463bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:45.640028 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94df83-537f-49be-acdd-2d2156463bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:45.640360 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94df83-537f-49be-acdd-2d2156463bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:45.641099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94df83-537f-49be-acdd-2d2156463bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:45.642734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94df83-537f-49be-acdd-2d2156463bb0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:45.654671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1182/2342] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:45.661389 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94df83-537f-49be-acdd-2d2156463bb0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:45.671555 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94df83-537f-49be-acdd-2d2156463bb0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:45.677283 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21da3d-e859-424a-a329-2bbc359f4cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:45.678127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21da3d-e859-424a-a329-2bbc359f4cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:45.678529 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21da3d-e859-424a-a329-2bbc359f4cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:45.679207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21da3d-e859-424a-a329-2bbc359f4cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:45.680018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21da3d-e859-424a-a329-2bbc359f4cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:45.682144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21da3d-e859-424a-a329-2bbc359f4cc5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:45.694622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1183/2343] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:45.698685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21da3d-e859-424a-a329-2bbc359f4cc5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:45.709849 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21da3d-e859-424a-a329-2bbc359f4cc5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:45.724796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d154f5-a17d-4079-a41e-0e7e27f6f6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:45.727716 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d154f5-a17d-4079-a41e-0e7e27f6f6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:45.727716 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d154f5-a17d-4079-a41e-0e7e27f6f6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:45.727716 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d154f5-a17d-4079-a41e-0e7e27f6f6d6 admin admin] PATH_INFO: `/v3/projects/cdcf4fbc9b5842bd80a0ff898a135260/users/229358997c1a4ef29494a9ea0012250e/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:45.744318 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d154f5-a17d-4079-a41e-0e7e27f6f6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdcf4fbc9b5842bd80a0ff898a135260, user_id=229358997c1a4ef29494a9ea0012250e, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:45.747775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d154f5-a17d-4079-a41e-0e7e27f6f6d6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:45.782099 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a6f56fdf-e69a-4348-95f2-3c7badc12c43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0433aa9c1fd468e8cebe6c55e9866e8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:45.798850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a6f56fdf-e69a-4348-95f2-3c7badc12c43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0433aa9c1fd468e8cebe6c55e9866e8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:45.803483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a6f56fdf-e69a-4348-95f2-3c7badc12c43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0433aa9c1fd468e8cebe6c55e9866e8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:45.827706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a6f56fdf-e69a-4348-95f2-3c7badc12c43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0433aa9c1fd468e8cebe6c55e9866e8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:45.827706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a6f56fdf-e69a-4348-95f2-3c7badc12c43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0433aa9c1fd468e8cebe6c55e9866e8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:45.867604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1184/2344] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:45 2022] PUT /identity/v3/projects/cdcf4fbc9b5842bd80a0ff898a135260/users/229358997c1a4ef29494a9ea0012250e/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:45.885890 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e4bdb-fb31-4c03-9e7e-faa8d5006bde None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:45.886182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e4bdb-fb31-4c03-9e7e-faa8d5006bde None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:45.886462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e4bdb-fb31-4c03-9e7e-faa8d5006bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:45.914383 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1161/2345] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:45 2022] DELETE /identity/v3/users/b0433aa9c1fd468e8cebe6c55e9866e8 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:45.916275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c31e4bdb-fb31-4c03-9e7e-faa8d5006bde None None] MFA Rules not processed for user `229358997c1a4ef29494a9ea0012250e`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:45.917635 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c31e4bdb-fb31-4c03-9e7e-faa8d5006bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:45.981658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:46.031114 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1185/2346] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:46.036531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f56fdf-e69a-4348-95f2-3c7badc12c43 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:46.038359 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a6f56fdf-e69a-4348-95f2-3c7badc12c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:46.121861 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f56fdf-e69a-4348-95f2-3c7badc12c43 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:46.126807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbcb79-1d34-4a96-bf98-a7761947fd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:46.127556 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbcb79-1d34-4a96-bf98-a7761947fd2d admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:46.127842 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbcb79-1d34-4a96-bf98-a7761947fd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:46.128309 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbcb79-1d34-4a96-bf98-a7761947fd2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:46.132947 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dbcb79-1d34-4a96-bf98-a7761947fd2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:46.132947 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dbcb79-1d34-4a96-bf98-a7761947fd2d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:46.232123 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1162/2347] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:46 2022] POST /identity/v3/projects => generated 381 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:46.233139 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e4bdb-fb31-4c03-9e7e-faa8d5006bde None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:46.253874 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e4bdb-fb31-4c03-9e7e-faa8d5006bde None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:46.336121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16cace-2707-4997-8282-6d273f51f3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:46.336857 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16cace-2707-4997-8282-6d273f51f3b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:46.337079 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16cace-2707-4997-8282-6d273f51f3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:46.337351 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16cace-2707-4997-8282-6d273f51f3b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:46.338059 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16cace-2707-4997-8282-6d273f51f3b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:46.347437 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16cace-2707-4997-8282-6d273f51f3b4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:46.552511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbcb79-1d34-4a96-bf98-a7761947fd2d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:46.553459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1186/2348] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:46 2022] POST /identity/v3/users => generated 297 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:46.557406 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-91dbcb79-1d34-4a96-bf98-a7761947fd2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:46.559396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16cace-2707-4997-8282-6d273f51f3b4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:46.582358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16cace-2707-4997-8282-6d273f51f3b4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:46.586261 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbcb79-1d34-4a96-bf98-a7761947fd2d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:46.590130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cb916-dba0-4820-92ee-21a698acb89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:46.590962 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cb916-dba0-4820-92ee-21a698acb89c admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:46.591267 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cb916-dba0-4820-92ee-21a698acb89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:46.591412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af32f6-6b46-43c4-a754-7f44e481231f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:46.591563 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cb916-dba0-4820-92ee-21a698acb89c admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:46.592257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cb916-dba0-4820-92ee-21a698acb89c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:46.592403 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af32f6-6b46-43c4-a754-7f44e481231f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:46.592578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af32f6-6b46-43c4-a754-7f44e481231f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:46.592861 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af32f6-6b46-43c4-a754-7f44e481231f admin admin] PATH_INFO: `/v3/projects/2075d37157814da2bc43dafdd72d1f9d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:46.593847 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cb916-dba0-4820-92ee-21a698acb89c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:46.594054 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af32f6-6b46-43c4-a754-7f44e481231f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2075d37157814da2bc43dafdd72d1f9d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:46.596018 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af32f6-6b46-43c4-a754-7f44e481231f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:46.607004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1187/2349] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:46.609708 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cb916-dba0-4820-92ee-21a698acb89c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:46.620487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cb916-dba0-4820-92ee-21a698acb89c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:46.625924 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c07fa-07b3-433d-b61f-75356afb0027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:46.626835 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c07fa-07b3-433d-b61f-75356afb0027 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:46.627262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c07fa-07b3-433d-b61f-75356afb0027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:46.627610 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c07fa-07b3-433d-b61f-75356afb0027 admin admin] PATH_INFO: `/v3/projects/ed0295a51f90408dbe668299471dd81c/users/16c5a642d69941fbbc8bf83fdc89781d/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:46.646737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c07fa-07b3-433d-b61f-75356afb0027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed0295a51f90408dbe668299471dd81c, user_id=16c5a642d69941fbbc8bf83fdc89781d, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:46.646737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c07fa-07b3-433d-b61f-75356afb0027 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:46.745747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1188/2350] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:46 2022] PUT /identity/v3/projects/ed0295a51f90408dbe668299471dd81c/users/16c5a642d69941fbbc8bf83fdc89781d/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:46.749588 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c07fa-07b3-433d-b61f-75356afb0027 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:46.754870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b1af32f6-6b46-43c4-a754-7f44e481231f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2075d37157814da2bc43dafdd72d1f9d', 'request_id': 'req-b1af32f6-6b46-43c4-a754-7f44e481231f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:46.761846 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b1af32f6-6b46-43c4-a754-7f44e481231f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2075d37157814da2bc43dafdd72d1f9d', 'request_id': 'req-b1af32f6-6b46-43c4-a754-7f44e481231f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:46.810123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c07fa-07b3-433d-b61f-75356afb0027 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:46.828610 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b1af32f6-6b46-43c4-a754-7f44e481231f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2075d37157814da2bc43dafdd72d1f9d', 'request_id': 'req-b1af32f6-6b46-43c4-a754-7f44e481231f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:46.843979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b1af32f6-6b46-43c4-a754-7f44e481231f admin admin] The token cache is being invalidate because project 2075d37157814da2bc43dafdd72d1f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:46.845228 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b1af32f6-6b46-43c4-a754-7f44e481231f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:46.846568 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b1af32f6-6b46-43c4-a754-7f44e481231f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:46.851472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1163/2351] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:46 2022] DELETE /identity/v3/projects/2075d37157814da2bc43dafdd72d1f9d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:46.860133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643dc180-4bb0-48bc-98c1-53029815102e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:46.860832 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643dc180-4bb0-48bc-98c1-53029815102e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:46.865415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643dc180-4bb0-48bc-98c1-53029815102e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:46.884359 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:46.889421 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27205a-c3d6-470b-a6b5-e49a75aa3807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:46.892762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27205a-c3d6-470b-a6b5-e49a75aa3807 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:46.892762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27205a-c3d6-470b-a6b5-e49a75aa3807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:46.892762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27205a-c3d6-470b-a6b5-e49a75aa3807 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:46.892762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27205a-c3d6-470b-a6b5-e49a75aa3807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:46.894850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27205a-c3d6-470b-a6b5-e49a75aa3807 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:46.906045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-643dc180-4bb0-48bc-98c1-53029815102e None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:46.908335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-643dc180-4bb0-48bc-98c1-53029815102e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:46.909145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1189/2352] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:46.917845 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27205a-c3d6-470b-a6b5-e49a75aa3807 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:46.917845 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ad27205a-c3d6-470b-a6b5-e49a75aa3807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:46.930460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1164/2353] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:46.934475 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27205a-c3d6-470b-a6b5-e49a75aa3807 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:46.942360 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b711687-3c35-4608-b50f-3ccf2d9122a7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:46.942666 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b711687-3c35-4608-b50f-3ccf2d9122a7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:46.942928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b711687-3c35-4608-b50f-3ccf2d9122a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:46.944954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43218a98-9194-4d8d-b794-34f09f33da67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:46.946088 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43218a98-9194-4d8d-b794-34f09f33da67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:46.946355 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43218a98-9194-4d8d-b794-34f09f33da67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:46.946634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43218a98-9194-4d8d-b794-34f09f33da67 admin admin] PATH_INFO: `/v3/projects/ed0295a51f90408dbe668299471dd81c/users/16c5a642d69941fbbc8bf83fdc89781d/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:46.951030 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43218a98-9194-4d8d-b794-34f09f33da67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed0295a51f90408dbe668299471dd81c, user_id=16c5a642d69941fbbc8bf83fdc89781d, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:46.953876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43218a98-9194-4d8d-b794-34f09f33da67 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:46.979677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8b711687-3c35-4608-b50f-3ccf2d9122a7 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:46.980931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1190/2354] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:46 2022] PUT /identity/v3/projects/ed0295a51f90408dbe668299471dd81c/users/16c5a642d69941fbbc8bf83fdc89781d/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:46.982071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8b711687-3c35-4608-b50f-3ccf2d9122a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:46.999282 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635cb28-9bb0-457b-8ba5-9fde29b31cf8 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:46.999671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635cb28-9bb0-457b-8ba5-9fde29b31cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:47.000048 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635cb28-9bb0-457b-8ba5-9fde29b31cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:47.048188 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6635cb28-9bb0-457b-8ba5-9fde29b31cf8 None None] MFA Rules not processed for user `16c5a642d69941fbbc8bf83fdc89781d`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:47.048188 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6635cb28-9bb0-457b-8ba5-9fde29b31cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:47.064482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1165/2355] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:47.074274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b711687-3c35-4608-b50f-3ccf2d9122a7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:47.083542 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b711687-3c35-4608-b50f-3ccf2d9122a7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:47.088748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0232c5-0295-4954-814e-a9fa7a9293df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:47.089424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0232c5-0295-4954-814e-a9fa7a9293df admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:47.089712 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0232c5-0295-4954-814e-a9fa7a9293df admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:47.090135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0232c5-0295-4954-814e-a9fa7a9293df admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:47.091399 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0232c5-0295-4954-814e-a9fa7a9293df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:47.091785 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0232c5-0295-4954-814e-a9fa7a9293df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:47.093421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0232c5-0295-4954-814e-a9fa7a9293df admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:47.110677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1166/2356] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:20:47.111742 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0232c5-0295-4954-814e-a9fa7a9293df admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:47.123301 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0232c5-0295-4954-814e-a9fa7a9293df admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:47.130960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d27e99-c49a-479c-9678-04d1bb28bf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:47.130960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d27e99-c49a-479c-9678-04d1bb28bf78 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:47.131233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1191/2357] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:47.131332 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d27e99-c49a-479c-9678-04d1bb28bf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:47.131421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d27e99-c49a-479c-9678-04d1bb28bf78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:47.132213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d27e99-c49a-479c-9678-04d1bb28bf78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:47.134186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d27e99-c49a-479c-9678-04d1bb28bf78 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:47.153556 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635cb28-9bb0-457b-8ba5-9fde29b31cf8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:47.155327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6635cb28-9bb0-457b-8ba5-9fde29b31cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:47.214702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635cb28-9bb0-457b-8ba5-9fde29b31cf8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:47.218398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f31f2-9918-4e88-b640-5375be0cce04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:47.219044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f31f2-9918-4e88-b640-5375be0cce04 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:47.219369 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f31f2-9918-4e88-b640-5375be0cce04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:47.219648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f31f2-9918-4e88-b640-5375be0cce04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:47.230006 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f31f2-9918-4e88-b640-5375be0cce04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:47.231279 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f31f2-9918-4e88-b640-5375be0cce04 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:47.241796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1192/2358] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:47.257149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1167/2359] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:47 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:47.261654 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f31f2-9918-4e88-b640-5375be0cce04 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:47.272694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f31f2-9918-4e88-b640-5375be0cce04 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:47.333974 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cdf11-8b3a-436f-81f0-b082e3031505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:47.334653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cdf11-8b3a-436f-81f0-b082e3031505 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:47.334945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cdf11-8b3a-436f-81f0-b082e3031505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:47.338785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cdf11-8b3a-436f-81f0-b082e3031505 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:47.338785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cdf11-8b3a-436f-81f0-b082e3031505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:47.338785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cdf11-8b3a-436f-81f0-b082e3031505 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:47.445989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1193/2360] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:47 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:47.451302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d27e99-c49a-479c-9678-04d1bb28bf78 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:47.462393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d27e99-c49a-479c-9678-04d1bb28bf78 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:47.468028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7703b-a7be-42b6-831c-6f85a611a88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:47.470842 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7703b-a7be-42b6-831c-6f85a611a88e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:47.470842 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7703b-a7be-42b6-831c-6f85a611a88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:47.470842 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7703b-a7be-42b6-831c-6f85a611a88e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:47.470842 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7703b-a7be-42b6-831c-6f85a611a88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:47.472296 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7703b-a7be-42b6-831c-6f85a611a88e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:47.484349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1168/2361] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:47.488840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cdf11-8b3a-436f-81f0-b082e3031505 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:47.499353 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cdf11-8b3a-436f-81f0-b082e3031505 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:47.505728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c8e92-c612-4cdc-b2c6-532abd5310fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:47.506581 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c8e92-c612-4cdc-b2c6-532abd5310fa admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:47.506861 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c8e92-c612-4cdc-b2c6-532abd5310fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:47.507150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c8e92-c612-4cdc-b2c6-532abd5310fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:47.507927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c8e92-c612-4cdc-b2c6-532abd5310fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:47.509756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c8e92-c612-4cdc-b2c6-532abd5310fa admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:47.523422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1194/2362] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:47.526602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7703b-a7be-42b6-831c-6f85a611a88e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:47.539230 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7703b-a7be-42b6-831c-6f85a611a88e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:47.553615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574a706-96ff-4725-a72d-a37e56db45ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:47.553615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574a706-96ff-4725-a72d-a37e56db45ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:47.553615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574a706-96ff-4725-a72d-a37e56db45ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:47.553615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574a706-96ff-4725-a72d-a37e56db45ef admin admin] PATH_INFO: `/v3/projects/a7676b5fcbc04076b2e9d0e36bbb07f1/users/b068a0e72d9e409aac407e3821649fb9/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:47.575729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5574a706-96ff-4725-a72d-a37e56db45ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7676b5fcbc04076b2e9d0e36bbb07f1, user_id=b068a0e72d9e409aac407e3821649fb9, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:47.581754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5574a706-96ff-4725-a72d-a37e56db45ef admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:47.640992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1169/2363] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:47 2022] PUT /identity/v3/projects/a7676b5fcbc04076b2e9d0e36bbb07f1/users/b068a0e72d9e409aac407e3821649fb9/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:47.646612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559a183-5427-4319-b65f-4961ca2f9382 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:47.646909 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559a183-5427-4319-b65f-4961ca2f9382 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:47.647262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559a183-5427-4319-b65f-4961ca2f9382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:47.681627 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c559a183-5427-4319-b65f-4961ca2f9382 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:47.683889 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c559a183-5427-4319-b65f-4961ca2f9382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:47.722697 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:47.795566 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1195/2364] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:47.807969 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574a706-96ff-4725-a72d-a37e56db45ef admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:47.810532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5574a706-96ff-4725-a72d-a37e56db45ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:47.837681 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:47.889141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574a706-96ff-4725-a72d-a37e56db45ef admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:47.892914 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51aa31-319b-484d-9ee8-f8c78fb3398f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:47.893599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51aa31-319b-484d-9ee8-f8c78fb3398f service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:47.893880 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51aa31-319b-484d-9ee8-f8c78fb3398f service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:47.894206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51aa31-319b-484d-9ee8-f8c78fb3398f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:47.911483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51aa31-319b-484d-9ee8-f8c78fb3398f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:47.912698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51aa31-319b-484d-9ee8-f8c78fb3398f service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:47.925950 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1170/2365] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:47.930414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559a183-5427-4319-b65f-4961ca2f9382 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:47.949175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559a183-5427-4319-b65f-4961ca2f9382 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:47.959725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cca812-6726-465f-a412-15fd6adc03d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:47.960744 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cca812-6726-465f-a412-15fd6adc03d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:47.961230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cca812-6726-465f-a412-15fd6adc03d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:47.968037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cca812-6726-465f-a412-15fd6adc03d3 admin admin] PATH_INFO: `/v3/users/229358997c1a4ef29494a9ea0012250e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:47.968037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cca812-6726-465f-a412-15fd6adc03d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=229358997c1a4ef29494a9ea0012250e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:47.968037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cca812-6726-465f-a412-15fd6adc03d3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:48.210384 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e7cca812-6726-465f-a412-15fd6adc03d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '229358997c1a4ef29494a9ea0012250e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:48.229395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e7cca812-6726-465f-a412-15fd6adc03d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '229358997c1a4ef29494a9ea0012250e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:48.305339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e7cca812-6726-465f-a412-15fd6adc03d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '229358997c1a4ef29494a9ea0012250e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:48.306301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e7cca812-6726-465f-a412-15fd6adc03d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '229358997c1a4ef29494a9ea0012250e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:48.322735 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e7cca812-6726-465f-a412-15fd6adc03d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '229358997c1a4ef29494a9ea0012250e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:48.332534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1196/2366] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:47 2022] DELETE /identity/v3/users/229358997c1a4ef29494a9ea0012250e => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:48.336977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e6b62-2d20-4102-8be3-fc7fc8d31fb7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:48.341149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e6b62-2d20-4102-8be3-fc7fc8d31fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:48.342224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e6b62-2d20-4102-8be3-fc7fc8d31fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:48.369704 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d24e6b62-2d20-4102-8be3-fc7fc8d31fb7 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:48.371532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d24e6b62-2d20-4102-8be3-fc7fc8d31fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:48.451688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1171/2367] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:48.464702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cca812-6726-465f-a412-15fd6adc03d3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:48.466635 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e7cca812-6726-465f-a412-15fd6adc03d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:48.488577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e6b62-2d20-4102-8be3-fc7fc8d31fb7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:48.490565 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d24e6b62-2d20-4102-8be3-fc7fc8d31fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:48.506039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e6b62-2d20-4102-8be3-fc7fc8d31fb7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:48.511266 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0cfdb-7e91-448f-9f00-6f386fff7fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:48.512191 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0cfdb-7e91-448f-9f00-6f386fff7fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:48.512478 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0cfdb-7e91-448f-9f00-6f386fff7fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:48.512928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0cfdb-7e91-448f-9f00-6f386fff7fa9 admin admin] PATH_INFO: `/v3/users/4f828fa6c46c4816945ef2cd77a33e8c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:48.514127 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0cfdb-7e91-448f-9f00-6f386fff7fa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f828fa6c46c4816945ef2cd77a33e8c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:48.516181 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0cfdb-7e91-448f-9f00-6f386fff7fa9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:48.536514 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cca812-6726-465f-a412-15fd6adc03d3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:48.540715 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8d46d-d07d-4047-b799-76fe4d0a343f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:48.541758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8d46d-d07d-4047-b799-76fe4d0a343f service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:48.542199 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8d46d-d07d-4047-b799-76fe4d0a343f service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:48.542638 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8d46d-d07d-4047-b799-76fe4d0a343f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:48.554072 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8d46d-d07d-4047-b799-76fe4d0a343f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:48.555649 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8d46d-d07d-4047-b799-76fe4d0a343f service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:48.566299 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1197/2368] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:48.684769 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-61b0cfdb-7e91-448f-9f00-6f386fff7fa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f828fa6c46c4816945ef2cd77a33e8c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:48.698619 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-61b0cfdb-7e91-448f-9f00-6f386fff7fa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f828fa6c46c4816945ef2cd77a33e8c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:48.699409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-61b0cfdb-7e91-448f-9f00-6f386fff7fa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f828fa6c46c4816945ef2cd77a33e8c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:48.707571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-61b0cfdb-7e91-448f-9f00-6f386fff7fa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f828fa6c46c4816945ef2cd77a33e8c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:48.708344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-61b0cfdb-7e91-448f-9f00-6f386fff7fa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f828fa6c46c4816945ef2cd77a33e8c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:48.781354 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:48.795238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1172/2369] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:48 2022] DELETE /identity/v3/users/4f828fa6c46c4816945ef2cd77a33e8c => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:20:48.801017 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72b1ba-c894-45e6-9e2d-c1ac408a69e6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:48.801310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72b1ba-c894-45e6-9e2d-c1ac408a69e6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:48.801947 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72b1ba-c894-45e6-9e2d-c1ac408a69e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:48.830322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5a72b1ba-c894-45e6-9e2d-c1ac408a69e6 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:48.833250 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5a72b1ba-c894-45e6-9e2d-c1ac408a69e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:48.871240 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:48.925058 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1198/2370] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:48.934407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0cfdb-7e91-448f-9f00-6f386fff7fa9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:48.936332 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-61b0cfdb-7e91-448f-9f00-6f386fff7fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:49.007899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0cfdb-7e91-448f-9f00-6f386fff7fa9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:49.011687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c5bbb-d69d-448c-9525-19ccbf6d717c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:49.012340 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c5bbb-d69d-448c-9525-19ccbf6d717c service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:49.012627 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c5bbb-d69d-448c-9525-19ccbf6d717c service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:49.012907 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c5bbb-d69d-448c-9525-19ccbf6d717c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:49.022995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c5bbb-d69d-448c-9525-19ccbf6d717c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:49.024471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c5bbb-d69d-448c-9525-19ccbf6d717c service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:49.037672 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1173/2371] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:49.069739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72b1ba-c894-45e6-9e2d-c1ac408a69e6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:49.071497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5a72b1ba-c894-45e6-9e2d-c1ac408a69e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:49.091050 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72b1ba-c894-45e6-9e2d-c1ac408a69e6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:49.112104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54db986-4333-4ec0-abc3-ff206e1a6d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:49.116327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54db986-4333-4ec0-abc3-ff206e1a6d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:49.116327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54db986-4333-4ec0-abc3-ff206e1a6d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:49.116327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54db986-4333-4ec0-abc3-ff206e1a6d27 admin admin] PATH_INFO: `/v3/projects/cdcf4fbc9b5842bd80a0ff898a135260` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:49.116327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54db986-4333-4ec0-abc3-ff206e1a6d27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdcf4fbc9b5842bd80a0ff898a135260)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:49.118734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54db986-4333-4ec0-abc3-ff206e1a6d27 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:49.267999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e54db986-4333-4ec0-abc3-ff206e1a6d27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdcf4fbc9b5842bd80a0ff898a135260', 'request_id': 'req-e54db986-4333-4ec0-abc3-ff206e1a6d27'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:49.274459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e54db986-4333-4ec0-abc3-ff206e1a6d27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdcf4fbc9b5842bd80a0ff898a135260', 'request_id': 'req-e54db986-4333-4ec0-abc3-ff206e1a6d27'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:49.398114 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e54db986-4333-4ec0-abc3-ff206e1a6d27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdcf4fbc9b5842bd80a0ff898a135260', 'request_id': 'req-e54db986-4333-4ec0-abc3-ff206e1a6d27'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:49.417376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e54db986-4333-4ec0-abc3-ff206e1a6d27 admin admin] The token cache is being invalidate because project cdcf4fbc9b5842bd80a0ff898a135260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:49.418129 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e54db986-4333-4ec0-abc3-ff206e1a6d27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:49.419161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e54db986-4333-4ec0-abc3-ff206e1a6d27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:49.426308 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c5bbb-d69d-448c-9525-19ccbf6d717c service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:49.427294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1199/2372] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:49 2022] DELETE /identity/v3/projects/cdcf4fbc9b5842bd80a0ff898a135260 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:49.438433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c5bbb-d69d-448c-9525-19ccbf6d717c service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:49.536929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c154f7-379b-4883-8fb7-27dfc5750ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:49.538068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c154f7-379b-4883-8fb7-27dfc5750ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:49.538463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c154f7-379b-4883-8fb7-27dfc5750ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:49.538843 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c154f7-379b-4883-8fb7-27dfc5750ae0 admin admin] PATH_INFO: `/v3/users/16c5a642d69941fbbc8bf83fdc89781d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:49.540816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c154f7-379b-4883-8fb7-27dfc5750ae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16c5a642d69941fbbc8bf83fdc89781d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:49.542469 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c154f7-379b-4883-8fb7-27dfc5750ae0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:49.598582 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:49.763521 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05c154f7-379b-4883-8fb7-27dfc5750ae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16c5a642d69941fbbc8bf83fdc89781d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:49.778945 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05c154f7-379b-4883-8fb7-27dfc5750ae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16c5a642d69941fbbc8bf83fdc89781d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:49.779744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05c154f7-379b-4883-8fb7-27dfc5750ae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16c5a642d69941fbbc8bf83fdc89781d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:49.789051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54db986-4333-4ec0-abc3-ff206e1a6d27 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:49.790702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e54db986-4333-4ec0-abc3-ff206e1a6d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:49.801651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05c154f7-379b-4883-8fb7-27dfc5750ae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16c5a642d69941fbbc8bf83fdc89781d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:49.805540 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05c154f7-379b-4883-8fb7-27dfc5750ae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16c5a642d69941fbbc8bf83fdc89781d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:49.808411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54db986-4333-4ec0-abc3-ff206e1a6d27 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:49.813573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49847df-7a8f-4b92-b1b3-8b7b9b03b2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:49.817669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49847df-7a8f-4b92-b1b3-8b7b9b03b2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:49.817669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49847df-7a8f-4b92-b1b3-8b7b9b03b2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:49.817669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49847df-7a8f-4b92-b1b3-8b7b9b03b2e1 admin admin] PATH_INFO: `/v3/projects/176337a3913047eb8d63d085b8a5db45` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:49.817669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49847df-7a8f-4b92-b1b3-8b7b9b03b2e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=176337a3913047eb8d63d085b8a5db45)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:49.819291 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49847df-7a8f-4b92-b1b3-8b7b9b03b2e1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:49.940189 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1174/2373] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:49 2022] DELETE /identity/v3/users/16c5a642d69941fbbc8bf83fdc89781d => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:49.970645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c0ed2-6190-4887-908b-476dc3f4498e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:49.971018 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c0ed2-6190-4887-908b-476dc3f4498e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:49.971393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c0ed2-6190-4887-908b-476dc3f4498e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:49.982727 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c49847df-7a8f-4b92-b1b3-8b7b9b03b2e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '176337a3913047eb8d63d085b8a5db45', 'request_id': 'req-c49847df-7a8f-4b92-b1b3-8b7b9b03b2e1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:49.990607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c49847df-7a8f-4b92-b1b3-8b7b9b03b2e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '176337a3913047eb8d63d085b8a5db45', 'request_id': 'req-c49847df-7a8f-4b92-b1b3-8b7b9b03b2e1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:50.021925 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ef2c0ed2-6190-4887-908b-476dc3f4498e None None] MFA Rules not processed for user `b068a0e72d9e409aac407e3821649fb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:50.021925 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ef2c0ed2-6190-4887-908b-476dc3f4498e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:50.070219 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c49847df-7a8f-4b92-b1b3-8b7b9b03b2e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '176337a3913047eb8d63d085b8a5db45', 'request_id': 'req-c49847df-7a8f-4b92-b1b3-8b7b9b03b2e1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:50.085133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c49847df-7a8f-4b92-b1b3-8b7b9b03b2e1 admin admin] The token cache is being invalidate because project 176337a3913047eb8d63d085b8a5db45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:50.085426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c49847df-7a8f-4b92-b1b3-8b7b9b03b2e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:50.086125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c49847df-7a8f-4b92-b1b3-8b7b9b03b2e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:50.088175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1200/2374] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:49 2022] DELETE /identity/v3/projects/176337a3913047eb8d63d085b8a5db45 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:50.093158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a52a2-ee72-4db2-b7a6-9159f65315ae None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:50.094052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a52a2-ee72-4db2-b7a6-9159f65315ae None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:50.094350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a52a2-ee72-4db2-b7a6-9159f65315ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:50.115732 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a38a52a2-ee72-4db2-b7a6-9159f65315ae None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:50.117462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a38a52a2-ee72-4db2-b7a6-9159f65315ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:50.141270 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1175/2375] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:50.146757 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c0ed2-6190-4887-908b-476dc3f4498e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:50.148390 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ef2c0ed2-6190-4887-908b-476dc3f4498e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:50.234968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1201/2376] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:50.244991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e22968-f36a-4d26-ad4f-7ca823423e5e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:50.245337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e22968-f36a-4d26-ad4f-7ca823423e5e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:50.245813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e22968-f36a-4d26-ad4f-7ca823423e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:50.263771 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c0ed2-6190-4887-908b-476dc3f4498e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:50.269300 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ac661-3cb8-4b17-802d-5d46803ec322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:50.270931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ac661-3cb8-4b17-802d-5d46803ec322 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:50.271355 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ac661-3cb8-4b17-802d-5d46803ec322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:50.271907 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ac661-3cb8-4b17-802d-5d46803ec322 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:50.272800 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ac661-3cb8-4b17-802d-5d46803ec322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:50.274780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e7e22968-f36a-4d26-ad4f-7ca823423e5e None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:50.275116 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ac661-3cb8-4b17-802d-5d46803ec322 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:50.276640 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e7e22968-f36a-4d26-ad4f-7ca823423e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:50.290394 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1202/2377] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:50.294786 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e22968-f36a-4d26-ad4f-7ca823423e5e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:50.304532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e22968-f36a-4d26-ad4f-7ca823423e5e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:50.309255 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e178bb4-cf5a-4e6d-801c-ccb17be3a838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:50.309988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e178bb4-cf5a-4e6d-801c-ccb17be3a838 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:50.310276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e178bb4-cf5a-4e6d-801c-ccb17be3a838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:50.310550 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e178bb4-cf5a-4e6d-801c-ccb17be3a838 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:50.311343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e178bb4-cf5a-4e6d-801c-ccb17be3a838 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:50.311648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e178bb4-cf5a-4e6d-801c-ccb17be3a838 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:50.313236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e178bb4-cf5a-4e6d-801c-ccb17be3a838 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:50.326147 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1203/2378] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:20:50.333064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e178bb4-cf5a-4e6d-801c-ccb17be3a838 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:50.343572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e178bb4-cf5a-4e6d-801c-ccb17be3a838 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:50.349311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc3466-c772-4f22-84d7-971c6b12a92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:50.350170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc3466-c772-4f22-84d7-971c6b12a92b admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:50.350562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc3466-c772-4f22-84d7-971c6b12a92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:50.351415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc3466-c772-4f22-84d7-971c6b12a92b admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:50.352180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc3466-c772-4f22-84d7-971c6b12a92b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:50.354171 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc3466-c772-4f22-84d7-971c6b12a92b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:50.387941 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1176/2379] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:50 2022] POST /identity/v3/projects => generated 371 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:50.391473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ac661-3cb8-4b17-802d-5d46803ec322 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:50.434571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:50.464843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1204/2380] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:50 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:50.472694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ac661-3cb8-4b17-802d-5d46803ec322 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:50.473134 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc3466-c772-4f22-84d7-971c6b12a92b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:50.563491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd4385-921b-46a9-9550-414e49012be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:50.564259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd4385-921b-46a9-9550-414e49012be3 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:50.564591 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd4385-921b-46a9-9550-414e49012be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:50.565037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd4385-921b-46a9-9550-414e49012be3 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:50.566252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd4385-921b-46a9-9550-414e49012be3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:50.568616 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd4385-921b-46a9-9550-414e49012be3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:50.571891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc3466-c772-4f22-84d7-971c6b12a92b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:50.577212 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77f64d-8e67-4001-8241-1e4f8a2d3a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:50.577925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77f64d-8e67-4001-8241-1e4f8a2d3a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:50.578273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77f64d-8e67-4001-8241-1e4f8a2d3a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:50.578666 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77f64d-8e67-4001-8241-1e4f8a2d3a31 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:50.580765 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77f64d-8e67-4001-8241-1e4f8a2d3a31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:50.582166 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77f64d-8e67-4001-8241-1e4f8a2d3a31 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:50.668115 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1177/2381] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:50 2022] POST /identity/v3/users => generated 292 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:50.669005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd4385-921b-46a9-9550-414e49012be3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:50.671346 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1205/2382] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:50 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:50.671483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-05dd4385-921b-46a9-9550-414e49012be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:50.672229 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77f64d-8e67-4001-8241-1e4f8a2d3a31 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:50.683079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd4385-921b-46a9-9550-414e49012be3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:50.685311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77f64d-8e67-4001-8241-1e4f8a2d3a31 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:50.696172 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc685f5c-153c-42ce-9f09-38601acf25de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:50.701889 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24530792-e44a-49d8-859a-62122b842bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:50.702124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc685f5c-153c-42ce-9f09-38601acf25de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:50.702124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc685f5c-153c-42ce-9f09-38601acf25de admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:50.702318 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24530792-e44a-49d8-859a-62122b842bae admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:50.702409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc685f5c-153c-42ce-9f09-38601acf25de admin admin] PATH_INFO: `/v3/projects/ed0295a51f90408dbe668299471dd81c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:50.702496 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24530792-e44a-49d8-859a-62122b842bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:50.702496 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24530792-e44a-49d8-859a-62122b842bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:50.702650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc685f5c-153c-42ce-9f09-38601acf25de admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed0295a51f90408dbe668299471dd81c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:50.702745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24530792-e44a-49d8-859a-62122b842bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:50.702745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24530792-e44a-49d8-859a-62122b842bae admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:50.702905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc685f5c-153c-42ce-9f09-38601acf25de admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:50.717625 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1206/2383] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:50.717625 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24530792-e44a-49d8-859a-62122b842bae admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:50.726367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24530792-e44a-49d8-859a-62122b842bae admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:50.737664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffb5eb-7512-4349-8f90-76ebccb86eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:50.737664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffb5eb-7512-4349-8f90-76ebccb86eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:50.737664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffb5eb-7512-4349-8f90-76ebccb86eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:50.738446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffb5eb-7512-4349-8f90-76ebccb86eb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:50.741052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fffb5eb-7512-4349-8f90-76ebccb86eb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:50.743089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fffb5eb-7512-4349-8f90-76ebccb86eb0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:50.755203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1207/2384] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:50.756051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffb5eb-7512-4349-8f90-76ebccb86eb0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:50.765767 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffb5eb-7512-4349-8f90-76ebccb86eb0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:50.770487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612f363-77d6-47af-94b1-d1364703b813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:50.771531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612f363-77d6-47af-94b1-d1364703b813 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:50.771836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612f363-77d6-47af-94b1-d1364703b813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:50.772977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612f363-77d6-47af-94b1-d1364703b813 admin admin] PATH_INFO: `/v3/projects/91e890bdd44142efabbe316506ef1a57/users/93fc52df31f446bcb7bfae01833adfc6/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:50.788386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0612f363-77d6-47af-94b1-d1364703b813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e890bdd44142efabbe316506ef1a57, user_id=93fc52df31f446bcb7bfae01833adfc6, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:50.792664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0612f363-77d6-47af-94b1-d1364703b813 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:50.908304 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1208/2385] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:50 2022] PUT /identity/v3/projects/91e890bdd44142efabbe316506ef1a57/users/93fc52df31f446bcb7bfae01833adfc6/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:50.908871 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612f363-77d6-47af-94b1-d1364703b813 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:50.919131 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cc685f5c-153c-42ce-9f09-38601acf25de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed0295a51f90408dbe668299471dd81c', 'request_id': 'req-cc685f5c-153c-42ce-9f09-38601acf25de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:50.928122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cc685f5c-153c-42ce-9f09-38601acf25de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed0295a51f90408dbe668299471dd81c', 'request_id': 'req-cc685f5c-153c-42ce-9f09-38601acf25de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:50.984684 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612f363-77d6-47af-94b1-d1364703b813 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:50.990230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31680595-a46c-406c-9c07-b9100cba67c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:50.990371 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cc685f5c-153c-42ce-9f09-38601acf25de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed0295a51f90408dbe668299471dd81c', 'request_id': 'req-cc685f5c-153c-42ce-9f09-38601acf25de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:50.990796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31680595-a46c-406c-9c07-b9100cba67c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:50.991058 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31680595-a46c-406c-9c07-b9100cba67c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:50.991385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31680595-a46c-406c-9c07-b9100cba67c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:50.992101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31680595-a46c-406c-9c07-b9100cba67c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:50.996125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31680595-a46c-406c-9c07-b9100cba67c2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:51.005741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cc685f5c-153c-42ce-9f09-38601acf25de admin admin] The token cache is being invalidate because project ed0295a51f90408dbe668299471dd81c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:51.007324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cc685f5c-153c-42ce-9f09-38601acf25de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:51.008053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cc685f5c-153c-42ce-9f09-38601acf25de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:51.010189 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1178/2386] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:50 2022] DELETE /identity/v3/projects/ed0295a51f90408dbe668299471dd81c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:51.010886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc685f5c-153c-42ce-9f09-38601acf25de admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:51.016509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-cc685f5c-153c-42ce-9f09-38601acf25de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:51.024676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1209/2387] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:51.025951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df522f2-94c0-486b-8ebc-31d75db89a90 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:51.026244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df522f2-94c0-486b-8ebc-31d75db89a90 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:51.026530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df522f2-94c0-486b-8ebc-31d75db89a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:51.055032 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3df522f2-94c0-486b-8ebc-31d75db89a90 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:51.056352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3df522f2-94c0-486b-8ebc-31d75db89a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:51.074776 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1210/2388] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:51.074776 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df522f2-94c0-486b-8ebc-31d75db89a90 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:51.074776 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3df522f2-94c0-486b-8ebc-31d75db89a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:51.081877 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df522f2-94c0-486b-8ebc-31d75db89a90 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:51.087494 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d88123-7736-4651-b206-73bb759d3d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:51.088527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d88123-7736-4651-b206-73bb759d3d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:51.088821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d88123-7736-4651-b206-73bb759d3d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:51.089099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d88123-7736-4651-b206-73bb759d3d17 admin admin] PATH_INFO: `/v3/projects/90288e11d18a4c58b77d848fdd630535/users/2321d0531bba403eb552c3a1dbb7484c/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:51.105178 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d88123-7736-4651-b206-73bb759d3d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90288e11d18a4c58b77d848fdd630535, user_id=2321d0531bba403eb552c3a1dbb7484c, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:51.107801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d88123-7736-4651-b206-73bb759d3d17 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:51.116793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc685f5c-153c-42ce-9f09-38601acf25de admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:51.121893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8dc42-6efe-4dc3-aee0-37a1a79cf6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:51.122897 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8dc42-6efe-4dc3-aee0-37a1a79cf6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:51.123200 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8dc42-6efe-4dc3-aee0-37a1a79cf6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:51.123479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8dc42-6efe-4dc3-aee0-37a1a79cf6c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:51.124199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8dc42-6efe-4dc3-aee0-37a1a79cf6c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:51.125782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8dc42-6efe-4dc3-aee0-37a1a79cf6c6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:51.136965 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1179/2389] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:51.138215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cff8a6-424f-4c4b-93a8-1b0cec3c87ab None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:51.138525 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cff8a6-424f-4c4b-93a8-1b0cec3c87ab None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:51.138812 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cff8a6-424f-4c4b-93a8-1b0cec3c87ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:51.159562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c0cff8a6-424f-4c4b-93a8-1b0cec3c87ab None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:51.160242 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c0cff8a6-424f-4c4b-93a8-1b0cec3c87ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:51.168874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1180/2390] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:51.172322 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cff8a6-424f-4c4b-93a8-1b0cec3c87ab None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:51.192867 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cff8a6-424f-4c4b-93a8-1b0cec3c87ab None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:51.195814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899440b-7240-4723-bc81-d0ad75248d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:51.196792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899440b-7240-4723-bc81-d0ad75248d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:51.197151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899440b-7240-4723-bc81-d0ad75248d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:51.197435 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899440b-7240-4723-bc81-d0ad75248d4b admin admin] PATH_INFO: `/v3/projects/91e890bdd44142efabbe316506ef1a57/users/93fc52df31f446bcb7bfae01833adfc6/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:51.199650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899440b-7240-4723-bc81-d0ad75248d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e890bdd44142efabbe316506ef1a57, user_id=93fc52df31f446bcb7bfae01833adfc6, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:51.202418 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899440b-7240-4723-bc81-d0ad75248d4b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:51.213492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1181/2391] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:51 2022] PUT /identity/v3/projects/91e890bdd44142efabbe316506ef1a57/users/93fc52df31f446bcb7bfae01833adfc6/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:51.214408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899440b-7240-4723-bc81-d0ad75248d4b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:51.232855 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:51.261215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1211/2392] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:51 2022] PUT /identity/v3/projects/90288e11d18a4c58b77d848fdd630535/users/2321d0531bba403eb552c3a1dbb7484c/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:51.279056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27ab12-d4ef-4c6b-8c2a-2939da64032c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:51.281777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27ab12-d4ef-4c6b-8c2a-2939da64032c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:51.282280 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27ab12-d4ef-4c6b-8c2a-2939da64032c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:51.306324 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0f27ab12-d4ef-4c6b-8c2a-2939da64032c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:51.307764 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0f27ab12-d4ef-4c6b-8c2a-2939da64032c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:51.314825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899440b-7240-4723-bc81-d0ad75248d4b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:51.383487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1212/2393] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:51.387912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bcdbb-ed99-44f5-9be4-75f4cc6ffa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:51.388590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bcdbb-ed99-44f5-9be4-75f4cc6ffa16 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:51.388872 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bcdbb-ed99-44f5-9be4-75f4cc6ffa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:51.389272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bcdbb-ed99-44f5-9be4-75f4cc6ffa16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:51.390010 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bcdbb-ed99-44f5-9be4-75f4cc6ffa16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:51.390318 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bcdbb-ed99-44f5-9be4-75f4cc6ffa16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:51.390868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27ab12-d4ef-4c6b-8c2a-2939da64032c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:51.392154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bcdbb-ed99-44f5-9be4-75f4cc6ffa16 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:51.392405 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0f27ab12-d4ef-4c6b-8c2a-2939da64032c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:51.405590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1182/2394] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:20:51.428835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bcdbb-ed99-44f5-9be4-75f4cc6ffa16 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:51.443807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bcdbb-ed99-44f5-9be4-75f4cc6ffa16 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:51.452079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9bc17-bf54-4b49-ba65-657d3db9d978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:51.452771 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9bc17-bf54-4b49-ba65-657d3db9d978 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:51.453062 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9bc17-bf54-4b49-ba65-657d3db9d978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:51.453492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9bc17-bf54-4b49-ba65-657d3db9d978 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:51.454620 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9bc17-bf54-4b49-ba65-657d3db9d978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:51.457377 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9bc17-bf54-4b49-ba65-657d3db9d978 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:51.467385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27ab12-d4ef-4c6b-8c2a-2939da64032c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:51.472978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234758e2-fa11-481d-ae80-9a51275901ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:51.473641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234758e2-fa11-481d-ae80-9a51275901ba service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:51.473925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234758e2-fa11-481d-ae80-9a51275901ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:51.474346 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234758e2-fa11-481d-ae80-9a51275901ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:51.490532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234758e2-fa11-481d-ae80-9a51275901ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:51.492173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234758e2-fa11-481d-ae80-9a51275901ba service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:51.503230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1213/2395] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:51.523967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234758e2-fa11-481d-ae80-9a51275901ba service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:51.525594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-234758e2-fa11-481d-ae80-9a51275901ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:51.540559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234758e2-fa11-481d-ae80-9a51275901ba service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:51.544631 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588e0d3-1e46-44b7-bf51-0d4cd0f22a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:51.545502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e0d3-1e46-44b7-bf51-0d4cd0f22a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:51.545788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e0d3-1e46-44b7-bf51-0d4cd0f22a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:51.546069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e0d3-1e46-44b7-bf51-0d4cd0f22a60 admin admin] PATH_INFO: `/v3/users/d721c3d69b2c4865ae23de3110c3c909` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:51.547307 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588e0d3-1e46-44b7-bf51-0d4cd0f22a60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d721c3d69b2c4865ae23de3110c3c909)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:51.549300 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588e0d3-1e46-44b7-bf51-0d4cd0f22a60 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:51.564812 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1183/2396] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:51 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:51.569443 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9bc17-bf54-4b49-ba65-657d3db9d978 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:51.659386 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9bc17-bf54-4b49-ba65-657d3db9d978 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:51.671576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399f2ad-e8a2-40d8-92d2-b0004c70cdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:51.672618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399f2ad-e8a2-40d8-92d2-b0004c70cdbc admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:51.677779 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399f2ad-e8a2-40d8-92d2-b0004c70cdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:51.678073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399f2ad-e8a2-40d8-92d2-b0004c70cdbc admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:51.679221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399f2ad-e8a2-40d8-92d2-b0004c70cdbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:51.681847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399f2ad-e8a2-40d8-92d2-b0004c70cdbc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:51.742671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6588e0d3-1e46-44b7-bf51-0d4cd0f22a60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd721c3d69b2c4865ae23de3110c3c909'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:51.755229 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6588e0d3-1e46-44b7-bf51-0d4cd0f22a60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd721c3d69b2c4865ae23de3110c3c909'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:51.815114 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1184/2397] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:51 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:51.817350 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399f2ad-e8a2-40d8-92d2-b0004c70cdbc admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:51.830507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6588e0d3-1e46-44b7-bf51-0d4cd0f22a60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd721c3d69b2c4865ae23de3110c3c909'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:51.832271 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6588e0d3-1e46-44b7-bf51-0d4cd0f22a60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd721c3d69b2c4865ae23de3110c3c909'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:51.835477 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399f2ad-e8a2-40d8-92d2-b0004c70cdbc admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:51.843559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6976383-3314-4100-885f-e8402c1e5f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:51.843559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6976383-3314-4100-885f-e8402c1e5f66 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:51.843559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6976383-3314-4100-885f-e8402c1e5f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:51.843854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6976383-3314-4100-885f-e8402c1e5f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:51.844619 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6588e0d3-1e46-44b7-bf51-0d4cd0f22a60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd721c3d69b2c4865ae23de3110c3c909'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:51.844877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6976383-3314-4100-885f-e8402c1e5f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:51.847718 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6976383-3314-4100-885f-e8402c1e5f66 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:51.853804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1214/2398] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:51 2022] DELETE /identity/v3/users/d721c3d69b2c4865ae23de3110c3c909 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:51.858895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659708e-19ec-42d2-a820-48e56645001c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:51.859894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659708e-19ec-42d2-a820-48e56645001c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:51.860190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659708e-19ec-42d2-a820-48e56645001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:51.872789 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1185/2399] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:20:51.876519 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6976383-3314-4100-885f-e8402c1e5f66 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:51.882646 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d6976383-3314-4100-885f-e8402c1e5f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:51.885079 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2659708e-19ec-42d2-a820-48e56645001c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:51.886318 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2659708e-19ec-42d2-a820-48e56645001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:51.907308 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6976383-3314-4100-885f-e8402c1e5f66 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:51.986496 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1215/2400] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:51.998134 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659708e-19ec-42d2-a820-48e56645001c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:52.000346 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2659708e-19ec-42d2-a820-48e56645001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:52.013535 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e6ba2-1bfe-4170-845f-a77ca4aea9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:52.014336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e6ba2-1bfe-4170-845f-a77ca4aea9d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:52.014579 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e6ba2-1bfe-4170-845f-a77ca4aea9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:52.014830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e6ba2-1bfe-4170-845f-a77ca4aea9d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:52.018458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e6ba2-1bfe-4170-845f-a77ca4aea9d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:52.018458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e6ba2-1bfe-4170-845f-a77ca4aea9d2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:52.031517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1186/2401] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:52.032547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e6ba2-1bfe-4170-845f-a77ca4aea9d2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:52.041816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:52.048938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e6ba2-1bfe-4170-845f-a77ca4aea9d2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:52.058923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35125324-1c94-4caf-8758-55ebffa85fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:52.060166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35125324-1c94-4caf-8758-55ebffa85fed admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:52.060447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35125324-1c94-4caf-8758-55ebffa85fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:52.060726 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35125324-1c94-4caf-8758-55ebffa85fed admin admin] PATH_INFO: `/v3/projects/c48bd22364774f4a96b96be02fe7495c/users/b9027b64b6714847be1006b712e741e9/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:52.081758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35125324-1c94-4caf-8758-55ebffa85fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c48bd22364774f4a96b96be02fe7495c, user_id=b9027b64b6714847be1006b712e741e9, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:52.084461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35125324-1c94-4caf-8758-55ebffa85fed admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:52.091867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659708e-19ec-42d2-a820-48e56645001c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:52.100118 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080ac12-59b7-4e12-9fcb-38f3e3478086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:52.100832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080ac12-59b7-4e12-9fcb-38f3e3478086 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:52.101180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080ac12-59b7-4e12-9fcb-38f3e3478086 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:52.101470 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080ac12-59b7-4e12-9fcb-38f3e3478086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:52.114213 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080ac12-59b7-4e12-9fcb-38f3e3478086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:52.115990 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080ac12-59b7-4e12-9fcb-38f3e3478086 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:52.128350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1216/2402] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:20:52.171804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1187/2403] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:52 2022] PUT /identity/v3/projects/c48bd22364774f4a96b96be02fe7495c/users/b9027b64b6714847be1006b712e741e9/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:52.194881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3699f-7e9e-444c-8b71-0010f434c75c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:52.195299 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3699f-7e9e-444c-8b71-0010f434c75c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:52.196084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3699f-7e9e-444c-8b71-0010f434c75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:52.243154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-76f3699f-7e9e-444c-8b71-0010f434c75c None None] MFA Rules not processed for user `b9027b64b6714847be1006b712e741e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:52.244667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-76f3699f-7e9e-444c-8b71-0010f434c75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:52.328854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1217/2404] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:52.342591 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35125324-1c94-4caf-8758-55ebffa85fed admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:52.345262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-35125324-1c94-4caf-8758-55ebffa85fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:52.434579 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35125324-1c94-4caf-8758-55ebffa85fed admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:52.445777 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b8447-bcf0-4326-a476-eb060c9533b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:52.446892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b8447-bcf0-4326-a476-eb060c9533b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:52.447671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b8447-bcf0-4326-a476-eb060c9533b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:52.447964 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b8447-bcf0-4326-a476-eb060c9533b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:52.469003 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b8447-bcf0-4326-a476-eb060c9533b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:52.471064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b8447-bcf0-4326-a476-eb060c9533b2 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:52.493297 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1188/2405] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:52.520668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3699f-7e9e-444c-8b71-0010f434c75c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:52.608879 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3699f-7e9e-444c-8b71-0010f434c75c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:52.621961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1193a7-ceba-420a-9aaa-8db40a165332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:52.623095 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1193a7-ceba-420a-9aaa-8db40a165332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:52.624668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1193a7-ceba-420a-9aaa-8db40a165332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:52.625254 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1193a7-ceba-420a-9aaa-8db40a165332 admin admin] PATH_INFO: `/v3/users/b9027b64b6714847be1006b712e741e9` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:52.629101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1193a7-ceba-420a-9aaa-8db40a165332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9027b64b6714847be1006b712e741e9)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:52.632425 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1193a7-ceba-420a-9aaa-8db40a165332 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:52.830480 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cac214-3f87-4e5f-9bdf-55e945f20f53 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:52.830683 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cac214-3f87-4e5f-9bdf-55e945f20f53 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:52.831578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cac214-3f87-4e5f-9bdf-55e945f20f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:52.862998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-54cac214-3f87-4e5f-9bdf-55e945f20f53 None None] MFA Rules not processed for user `93fc52df31f446bcb7bfae01833adfc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:52.864312 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-54cac214-3f87-4e5f-9bdf-55e945f20f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:52.937590 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3a1193a7-ceba-420a-9aaa-8db40a165332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9027b64b6714847be1006b712e741e9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:52.958657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3a1193a7-ceba-420a-9aaa-8db40a165332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9027b64b6714847be1006b712e741e9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:52.971981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1189/2406] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:52 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:52.973979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cac214-3f87-4e5f-9bdf-55e945f20f53 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:52.978210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-54cac214-3f87-4e5f-9bdf-55e945f20f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:52.980711 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:52.997845 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cac214-3f87-4e5f-9bdf-55e945f20f53 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:53.002666 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4527a0-52a8-400d-853b-e5be2604a111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:53.003977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4527a0-52a8-400d-853b-e5be2604a111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:53.004264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4527a0-52a8-400d-853b-e5be2604a111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:53.008053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4527a0-52a8-400d-853b-e5be2604a111 admin admin] PATH_INFO: `/v3/projects/ad86395efbb949d89e70d8f13f47ecc3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:53.008053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4527a0-52a8-400d-853b-e5be2604a111 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad86395efbb949d89e70d8f13f47ecc3)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:53.008053 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4527a0-52a8-400d-853b-e5be2604a111 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:53.020689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3a1193a7-ceba-420a-9aaa-8db40a165332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9027b64b6714847be1006b712e741e9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:53.021627 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3a1193a7-ceba-420a-9aaa-8db40a165332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9027b64b6714847be1006b712e741e9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:53.031514 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3a1193a7-ceba-420a-9aaa-8db40a165332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9027b64b6714847be1006b712e741e9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:53.038393 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1218/2407] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:20:52 2022] DELETE /identity/v3/users/b9027b64b6714847be1006b712e741e9 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:53.041915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1193a7-ceba-420a-9aaa-8db40a165332 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:53.043461 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3a1193a7-ceba-420a-9aaa-8db40a165332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:53.106700 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8f4527a0-52a8-400d-853b-e5be2604a111 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad86395efbb949d89e70d8f13f47ecc3', 'request_id': 'req-8f4527a0-52a8-400d-853b-e5be2604a111'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:53.114309 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1193a7-ceba-420a-9aaa-8db40a165332 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:53.116150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8f4527a0-52a8-400d-853b-e5be2604a111 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad86395efbb949d89e70d8f13f47ecc3', 'request_id': 'req-8f4527a0-52a8-400d-853b-e5be2604a111'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:53.162437 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239668a-ec4d-433e-9aa7-afee48e099c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:53.163084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239668a-ec4d-433e-9aa7-afee48e099c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:53.163401 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239668a-ec4d-433e-9aa7-afee48e099c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:53.163681 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239668a-ec4d-433e-9aa7-afee48e099c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:53.165783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1239668a-ec4d-433e-9aa7-afee48e099c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:53.222694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239668a-ec4d-433e-9aa7-afee48e099c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:53.224329 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239668a-ec4d-433e-9aa7-afee48e099c0 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:53.233062 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1219/2408] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:53.234455 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f127d4-11c6-48b7-aede-4b5a5e207c4b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:53.234668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f127d4-11c6-48b7-aede-4b5a5e207c4b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:53.234946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f127d4-11c6-48b7-aede-4b5a5e207c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:53.237535 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8f4527a0-52a8-400d-853b-e5be2604a111 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad86395efbb949d89e70d8f13f47ecc3', 'request_id': 'req-8f4527a0-52a8-400d-853b-e5be2604a111'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:53.256509 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e9f127d4-11c6-48b7-aede-4b5a5e207c4b None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:53.263414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e9f127d4-11c6-48b7-aede-4b5a5e207c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:53.274183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8f4527a0-52a8-400d-853b-e5be2604a111 admin admin] The token cache is being invalidate because project ad86395efbb949d89e70d8f13f47ecc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:53.274655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8f4527a0-52a8-400d-853b-e5be2604a111 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:53.275491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8f4527a0-52a8-400d-853b-e5be2604a111 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:53.278276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1190/2409] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:52 2022] DELETE /identity/v3/projects/ad86395efbb949d89e70d8f13f47ecc3 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:53.290653 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241620c-efa6-400a-a610-a5ad9ce6bc54 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:53.290954 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241620c-efa6-400a-a610-a5ad9ce6bc54 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:53.296028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241620c-efa6-400a-a610-a5ad9ce6bc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:53.343821 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1220/2410] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:53.344729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c241620c-efa6-400a-a610-a5ad9ce6bc54 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:53.345980 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c241620c-efa6-400a-a610-a5ad9ce6bc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:53.353803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f127d4-11c6-48b7-aede-4b5a5e207c4b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:53.355741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e9f127d4-11c6-48b7-aede-4b5a5e207c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:53.377048 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1191/2411] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:53.381757 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f875a2-f40b-402d-a443-3751319bc556 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:53.382050 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f875a2-f40b-402d-a443-3751319bc556 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:53.382330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f875a2-f40b-402d-a443-3751319bc556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:53.422663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-61f875a2-f40b-402d-a443-3751319bc556 None None] MFA Rules not processed for user `2321d0531bba403eb552c3a1dbb7484c`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:53.422663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-61f875a2-f40b-402d-a443-3751319bc556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:53.461546 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f127d4-11c6-48b7-aede-4b5a5e207c4b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:53.465549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80f407-ba40-4798-a8dc-4d783467b8ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:53.466638 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80f407-ba40-4798-a8dc-4d783467b8ec service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:53.467497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80f407-ba40-4798-a8dc-4d783467b8ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:53.467782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80f407-ba40-4798-a8dc-4d783467b8ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:53.480287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80f407-ba40-4798-a8dc-4d783467b8ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:53.481961 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80f407-ba40-4798-a8dc-4d783467b8ec service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:53.492800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1221/2412] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:53.494254 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd3ad5-9162-4e70-b260-cfb1e53aadc6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:53.494428 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd3ad5-9162-4e70-b260-cfb1e53aadc6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:53.494699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd3ad5-9162-4e70-b260-cfb1e53aadc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:53.520001 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-11fd3ad5-9162-4e70-b260-cfb1e53aadc6 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:53.521976 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-11fd3ad5-9162-4e70-b260-cfb1e53aadc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:53.524431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1192/2413] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:53.533461 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1222/2414] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:53.537497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd3ad5-9162-4e70-b260-cfb1e53aadc6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:53.539394 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f875a2-f40b-402d-a443-3751319bc556 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:53.541816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:53.543765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-61f875a2-f40b-402d-a443-3751319bc556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:53.566167 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f875a2-f40b-402d-a443-3751319bc556 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:53.567061 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd3ad5-9162-4e70-b260-cfb1e53aadc6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:53.571472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84accae3-122c-41e6-9a74-52a3e868d87a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:53.572331 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84accae3-122c-41e6-9a74-52a3e868d87a service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:53.572617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84accae3-122c-41e6-9a74-52a3e868d87a service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:53.575598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1855e6-0556-4c83-86cf-bd810dd7ac75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:53.575720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84accae3-122c-41e6-9a74-52a3e868d87a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:53.575802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1855e6-0556-4c83-86cf-bd810dd7ac75 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:53.575802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1855e6-0556-4c83-86cf-bd810dd7ac75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:53.575802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1855e6-0556-4c83-86cf-bd810dd7ac75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:53.575802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1855e6-0556-4c83-86cf-bd810dd7ac75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:53.575802 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1855e6-0556-4c83-86cf-bd810dd7ac75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:53.577070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1855e6-0556-4c83-86cf-bd810dd7ac75 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:53.584559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84accae3-122c-41e6-9a74-52a3e868d87a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:53.586045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84accae3-122c-41e6-9a74-52a3e868d87a service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:53.592262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1223/2415] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:20:53.599286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1193/2416] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:53.602467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1855e6-0556-4c83-86cf-bd810dd7ac75 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:53.614781 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1855e6-0556-4c83-86cf-bd810dd7ac75 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:53.624555 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1367a3-c5e0-47b6-847d-c7d7bb6de506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:53.628190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1367a3-c5e0-47b6-847d-c7d7bb6de506 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:53.628190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1367a3-c5e0-47b6-847d-c7d7bb6de506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:53.628190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1367a3-c5e0-47b6-847d-c7d7bb6de506 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:53.628190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1367a3-c5e0-47b6-847d-c7d7bb6de506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:53.636547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1367a3-c5e0-47b6-847d-c7d7bb6de506 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:53.769951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1224/2417] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:53 2022] POST /identity/v3/projects => generated 371 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:20:53.770768 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84accae3-122c-41e6-9a74-52a3e868d87a service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:53.799109 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84accae3-122c-41e6-9a74-52a3e868d87a service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:53.860571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:53.868672 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e39830-d608-4738-b7c9-2c30e9b556b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:53.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e39830-d608-4738-b7c9-2c30e9b556b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:53.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e39830-d608-4738-b7c9-2c30e9b556b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:53.870058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e39830-d608-4738-b7c9-2c30e9b556b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:53.870604 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e39830-d608-4738-b7c9-2c30e9b556b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:53.872590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e39830-d608-4738-b7c9-2c30e9b556b9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:53.964388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1194/2418] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:53 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:53.968593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1367a3-c5e0-47b6-847d-c7d7bb6de506 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:53.979299 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1367a3-c5e0-47b6-847d-c7d7bb6de506 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:53.984233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f7622-72a9-4c36-a876-f8538bc32a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:53.984931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f7622-72a9-4c36-a876-f8538bc32a58 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:53.985217 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f7622-72a9-4c36-a876-f8538bc32a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:53.985497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f7622-72a9-4c36-a876-f8538bc32a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:53.986229 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f7622-72a9-4c36-a876-f8538bc32a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:53.988834 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f7622-72a9-4c36-a876-f8538bc32a58 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:54.009882 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1225/2419] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:54.021974 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e39830-d608-4738-b7c9-2c30e9b556b9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:54.032679 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e39830-d608-4738-b7c9-2c30e9b556b9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:54.037772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357c027-151d-4087-936e-2ee10163c128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:54.038435 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357c027-151d-4087-936e-2ee10163c128 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:54.038725 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357c027-151d-4087-936e-2ee10163c128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:54.039008 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357c027-151d-4087-936e-2ee10163c128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:54.039771 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e357c027-151d-4087-936e-2ee10163c128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:54.041473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e357c027-151d-4087-936e-2ee10163c128 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:54.055279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1195/2420] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:54.059583 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f7622-72a9-4c36-a876-f8538bc32a58 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:54.059583 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-175f7622-72a9-4c36-a876-f8538bc32a58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:54.059763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357c027-151d-4087-936e-2ee10163c128 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:54.078148 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357c027-151d-4087-936e-2ee10163c128 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:54.079728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f7622-72a9-4c36-a876-f8538bc32a58 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:54.084071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72874ad-86a8-46b6-a3f5-3d2f74bad6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:54.087344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72874ad-86a8-46b6-a3f5-3d2f74bad6e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:54.087344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72874ad-86a8-46b6-a3f5-3d2f74bad6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:54.087582 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c211e7-e789-4581-9a35-8157132de246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:54.087699 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72874ad-86a8-46b6-a3f5-3d2f74bad6e4 admin admin] PATH_INFO: `/v3/projects/66edbd777b48472b8fcd5a3f43c68dcc/users/bb55f38fa3c741cbb7f00e310368a85b/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:54.087807 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c211e7-e789-4581-9a35-8157132de246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:54.087807 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c211e7-e789-4581-9a35-8157132de246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:54.087807 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c211e7-e789-4581-9a35-8157132de246 admin admin] PATH_INFO: `/v3/projects/c48bd22364774f4a96b96be02fe7495c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:54.090159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c211e7-e789-4581-9a35-8157132de246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c48bd22364774f4a96b96be02fe7495c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:54.092977 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c211e7-e789-4581-9a35-8157132de246 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:54.110549 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72874ad-86a8-46b6-a3f5-3d2f74bad6e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66edbd777b48472b8fcd5a3f43c68dcc, user_id=bb55f38fa3c741cbb7f00e310368a85b, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:54.114449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72874ad-86a8-46b6-a3f5-3d2f74bad6e4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:54.215330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1196/2421] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:54 2022] PUT /identity/v3/projects/66edbd777b48472b8fcd5a3f43c68dcc/users/bb55f38fa3c741cbb7f00e310368a85b/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:54.221031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6e1d6-75d6-4589-8d3c-50122f993499 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:54.221875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6e1d6-75d6-4589-8d3c-50122f993499 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:54.222216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6e1d6-75d6-4589-8d3c-50122f993499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:54.227113 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-62c211e7-e789-4581-9a35-8157132de246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c48bd22364774f4a96b96be02fe7495c', 'request_id': 'req-62c211e7-e789-4581-9a35-8157132de246'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:54.233706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-62c211e7-e789-4581-9a35-8157132de246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c48bd22364774f4a96b96be02fe7495c', 'request_id': 'req-62c211e7-e789-4581-9a35-8157132de246'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:54.239994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-05c6e1d6-75d6-4589-8d3c-50122f993499 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:54.241237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-05c6e1d6-75d6-4589-8d3c-50122f993499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:54.299115 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1197/2422] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:20:54.313118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6e1d6-75d6-4589-8d3c-50122f993499 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:54.314587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-05c6e1d6-75d6-4589-8d3c-50122f993499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:54.371151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6e1d6-75d6-4589-8d3c-50122f993499 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:54.375271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cc3e8-e5b9-41ee-854d-0412e96360e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:54.375670 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cc3e8-e5b9-41ee-854d-0412e96360e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:54.375954 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cc3e8-e5b9-41ee-854d-0412e96360e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:54.376292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cc3e8-e5b9-41ee-854d-0412e96360e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:54.386603 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cc3e8-e5b9-41ee-854d-0412e96360e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:54.388064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cc3e8-e5b9-41ee-854d-0412e96360e0 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:54.398101 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1198/2423] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:20:54.409788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-62c211e7-e789-4581-9a35-8157132de246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c48bd22364774f4a96b96be02fe7495c', 'request_id': 'req-62c211e7-e789-4581-9a35-8157132de246'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:54.435259 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-62c211e7-e789-4581-9a35-8157132de246 admin admin] The token cache is being invalidate because project c48bd22364774f4a96b96be02fe7495c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:20:54.435597 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-62c211e7-e789-4581-9a35-8157132de246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:54.436486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-62c211e7-e789-4581-9a35-8157132de246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:20:54.442803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1226/2424] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:20:54 2022] DELETE /identity/v3/projects/c48bd22364774f4a96b96be02fe7495c => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:54.447305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01897da-00e4-4623-859d-e863060f7bf1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:54.447602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01897da-00e4-4623-859d-e863060f7bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:54.447942 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01897da-00e4-4623-859d-e863060f7bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:54.468130 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e01897da-00e4-4623-859d-e863060f7bf1 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:54.471850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e01897da-00e4-4623-859d-e863060f7bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:54.478271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1199/2425] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:54.485810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffc80f-937a-4407-82a7-65a475c0c452 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:54.486105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffc80f-937a-4407-82a7-65a475c0c452 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:54.486437 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffc80f-937a-4407-82a7-65a475c0c452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:54.505910 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-01ffc80f-937a-4407-82a7-65a475c0c452 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:54.507115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-01ffc80f-937a-4407-82a7-65a475c0c452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:54.515287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1227/2426] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:54.520179 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01897da-00e4-4623-859d-e863060f7bf1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:54.530992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01897da-00e4-4623-859d-e863060f7bf1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:54.536560 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578efdd5-5c0b-4aa4-b4eb-35b73ce016ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:54.537228 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578efdd5-5c0b-4aa4-b4eb-35b73ce016ed admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:54.537914 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578efdd5-5c0b-4aa4-b4eb-35b73ce016ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:54.537914 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578efdd5-5c0b-4aa4-b4eb-35b73ce016ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:54.538531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578efdd5-5c0b-4aa4-b4eb-35b73ce016ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:20:54.538851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578efdd5-5c0b-4aa4-b4eb-35b73ce016ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:54.540476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578efdd5-5c0b-4aa4-b4eb-35b73ce016ed admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:54.552754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1200/2427] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:20:54.557073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffc80f-937a-4407-82a7-65a475c0c452 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:54.568690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffc80f-937a-4407-82a7-65a475c0c452 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:54.573943 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc43e5-9c50-4e31-9654-72820bdeded5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:54.574743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc43e5-9c50-4e31-9654-72820bdeded5 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:54.575007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc43e5-9c50-4e31-9654-72820bdeded5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:54.575474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc43e5-9c50-4e31-9654-72820bdeded5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:54.576390 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc43e5-9c50-4e31-9654-72820bdeded5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:54.578441 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc43e5-9c50-4e31-9654-72820bdeded5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:54.714666 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1228/2428] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:20:54 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:54.719052 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578efdd5-5c0b-4aa4-b4eb-35b73ce016ed admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:54.726624 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:54.732057 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578efdd5-5c0b-4aa4-b4eb-35b73ce016ed admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:54.792823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02169e-df34-4b74-8340-6029100c499a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:54.793531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02169e-df34-4b74-8340-6029100c499a admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:54.793820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02169e-df34-4b74-8340-6029100c499a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:54.794292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02169e-df34-4b74-8340-6029100c499a admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:54.794797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02169e-df34-4b74-8340-6029100c499a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:54.796975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02169e-df34-4b74-8340-6029100c499a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:54.888767 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1201/2429] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:20:54 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:20:54.892873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc43e5-9c50-4e31-9654-72820bdeded5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:54.904621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc43e5-9c50-4e31-9654-72820bdeded5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:54.909831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ed71b-9e3f-4db3-a098-4ff7057b324e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:54.910386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ed71b-9e3f-4db3-a098-4ff7057b324e admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:54.910675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ed71b-9e3f-4db3-a098-4ff7057b324e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:54.910964 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ed71b-9e3f-4db3-a098-4ff7057b324e admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:54.911766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ed71b-9e3f-4db3-a098-4ff7057b324e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:54.913452 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ed71b-9e3f-4db3-a098-4ff7057b324e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:54.928614 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1229/2430] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:54.929778 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02169e-df34-4b74-8340-6029100c499a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:54.940562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02169e-df34-4b74-8340-6029100c499a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:54.945428 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc61db-06a9-4039-b4fa-5a36a0a005a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:54.946094 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc61db-06a9-4039-b4fa-5a36a0a005a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:54.946376 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc61db-06a9-4039-b4fa-5a36a0a005a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:54.949468 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc61db-06a9-4039-b4fa-5a36a0a005a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:54.949468 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc61db-06a9-4039-b4fa-5a36a0a005a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:54.949468 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc61db-06a9-4039-b4fa-5a36a0a005a9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:54.959957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1202/2431] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:20:54.964332 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ed71b-9e3f-4db3-a098-4ff7057b324e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:54.974806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ed71b-9e3f-4db3-a098-4ff7057b324e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:54.979656 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b64de-c414-4977-b73f-c0242d112004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:54.980549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b64de-c414-4977-b73f-c0242d112004 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:54.980827 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b64de-c414-4977-b73f-c0242d112004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:54.981103 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b64de-c414-4977-b73f-c0242d112004 admin admin] PATH_INFO: `/v3/projects/fd17b9ad47244c20a6f61213ce1fb5ce/users/66f0325855124a8b89fbe13aa13dbf9c/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:54.996507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b64de-c414-4977-b73f-c0242d112004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd17b9ad47244c20a6f61213ce1fb5ce, user_id=66f0325855124a8b89fbe13aa13dbf9c, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:54.999170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b64de-c414-4977-b73f-c0242d112004 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:55.108917 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1230/2432] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:20:54 2022] PUT /identity/v3/projects/fd17b9ad47244c20a6f61213ce1fb5ce/users/66f0325855124a8b89fbe13aa13dbf9c/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:20:55.119543 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018b7bb-f8cb-4aec-af20-20354f2cf47c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:55.120001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018b7bb-f8cb-4aec-af20-20354f2cf47c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:55.120293 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018b7bb-f8cb-4aec-af20-20354f2cf47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:55.150685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e018b7bb-f8cb-4aec-af20-20354f2cf47c None None] MFA Rules not processed for user `66f0325855124a8b89fbe13aa13dbf9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:55.152010 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e018b7bb-f8cb-4aec-af20-20354f2cf47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:55.221532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1203/2433] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:55.232878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b64de-c414-4977-b73f-c0242d112004 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:55.234722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5d0b64de-c414-4977-b73f-c0242d112004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:55.305481 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b64de-c414-4977-b73f-c0242d112004 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:55.309875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a4bca-9903-4e9d-94e1-644240eae881 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:55.310204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a4bca-9903-4e9d-94e1-644240eae881 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:55.310509 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a4bca-9903-4e9d-94e1-644240eae881 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:55.310781 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a4bca-9903-4e9d-94e1-644240eae881 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:55.320806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a4bca-9903-4e9d-94e1-644240eae881 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:55.322424 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a4bca-9903-4e9d-94e1-644240eae881 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:55.331671 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1231/2434] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:55.415466 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:55.826101 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:56.113012 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae1fef-8a9c-438f-9af4-67170795a593 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:56.113449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae1fef-8a9c-438f-9af4-67170795a593 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:56.113914 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae1fef-8a9c-438f-9af4-67170795a593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:56.134652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-bfae1fef-8a9c-438f-9af4-67170795a593 None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:56.136339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bfae1fef-8a9c-438f-9af4-67170795a593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:56.187563 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a4bca-9903-4e9d-94e1-644240eae881 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:56.198473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a4bca-9903-4e9d-94e1-644240eae881 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:56.204918 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40040f02-c5ad-4c83-8c1a-e197de3dafc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:56.206426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40040f02-c5ad-4c83-8c1a-e197de3dafc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:56.206426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40040f02-c5ad-4c83-8c1a-e197de3dafc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:56.206709 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40040f02-c5ad-4c83-8c1a-e197de3dafc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:56.212126 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-40040f02-c5ad-4c83-8c1a-e197de3dafc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:56.222760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1204/2435] 10.222.0.67 () {58 vars in 1009 bytes} [Fri Jun 24 15:20:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:56.294797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae1fef-8a9c-438f-9af4-67170795a593 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:56.296218 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bfae1fef-8a9c-438f-9af4-67170795a593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:56.308231 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40040f02-c5ad-4c83-8c1a-e197de3dafc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:56.310404 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40040f02-c5ad-4c83-8c1a-e197de3dafc1 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:56.365029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1232/2436] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:56 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:20:56.416982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae1fef-8a9c-438f-9af4-67170795a593 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:56.427210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f18e7-8b67-439e-a5ab-88594d4d08c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:56.427986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f18e7-8b67-439e-a5ab-88594d4d08c8 service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:56.428333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f18e7-8b67-439e-a5ab-88594d4d08c8 service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:56.428673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f18e7-8b67-439e-a5ab-88594d4d08c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:56.438457 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f18e7-8b67-439e-a5ab-88594d4d08c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:56.440723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f18e7-8b67-439e-a5ab-88594d4d08c8 service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:56.455496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1205/2437] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:20:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 15:20:56.680199 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37e9fe-bc62-4ab6-93b2-4ed5cc12cbb7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:56.680759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37e9fe-bc62-4ab6-93b2-4ed5cc12cbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:56.681152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37e9fe-bc62-4ab6-93b2-4ed5cc12cbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:56.716454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7b37e9fe-bc62-4ab6-93b2-4ed5cc12cbb7 None None] MFA Rules not processed for user `2321d0531bba403eb552c3a1dbb7484c`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:56.722069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7b37e9fe-bc62-4ab6-93b2-4ed5cc12cbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:56.878323 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:56.878927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1233/2438] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:20:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:56.991251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:58.953171 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97301a3b-2308-49e7-b6fb-34764bcf0a7e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:58.953998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97301a3b-2308-49e7-b6fb-34764bcf0a7e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:58.953998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97301a3b-2308-49e7-b6fb-34764bcf0a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:59.064118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-97301a3b-2308-49e7-b6fb-34764bcf0a7e None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:20:59.065432 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-97301a3b-2308-49e7-b6fb-34764bcf0a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:59.139510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1206/2439] 10.222.0.67 () {58 vars in 1009 bytes} [Fri Jun 24 15:20:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:20:59.163269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37e9fe-bc62-4ab6-93b2-4ed5cc12cbb7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:59.355326 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37e9fe-bc62-4ab6-93b2-4ed5cc12cbb7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:59.371019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4787f-8d08-481c-a24f-67378b0e2765 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:59.381093 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4787f-8d08-481c-a24f-67378b0e2765 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:59.381093 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4787f-8d08-481c-a24f-67378b0e2765 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:59.381093 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4787f-8d08-481c-a24f-67378b0e2765 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:59.388166 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4787f-8d08-481c-a24f-67378b0e2765 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:59.390225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4787f-8d08-481c-a24f-67378b0e2765 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:59.421412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1234/2440] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:20:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:20:59.887511 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:00.014821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:00.870035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:01.277830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2b1fc-cc96-4c42-a7f2-84eee5b9ff7b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:01.278992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2b1fc-cc96-4c42-a7f2-84eee5b9ff7b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:01.279474 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2b1fc-cc96-4c42-a7f2-84eee5b9ff7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:01.319725 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0cb2b1fc-cc96-4c42-a7f2-84eee5b9ff7b None None] MFA Rules not processed for user `b67761f9b13340a090efc1cb07714912`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:01.322282 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0cb2b1fc-cc96-4c42-a7f2-84eee5b9ff7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:01.492168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1207/2441] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:02.615003 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:03.752493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:08.317891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24dd5e-2a2f-4b99-9063-20fcf15133d4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:08.317891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24dd5e-2a2f-4b99-9063-20fcf15133d4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:08.317891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24dd5e-2a2f-4b99-9063-20fcf15133d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:08.350211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1b24dd5e-2a2f-4b99-9063-20fcf15133d4 None None] MFA Rules not processed for user `bb55f38fa3c741cbb7f00e310368a85b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:08.352077 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1b24dd5e-2a2f-4b99-9063-20fcf15133d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:08.407080 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:08.493554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1235/2442] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:08.494773 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2b1fc-cc96-4c42-a7f2-84eee5b9ff7b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:08.496582 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0cb2b1fc-cc96-4c42-a7f2-84eee5b9ff7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:08.889415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2b1fc-cc96-4c42-a7f2-84eee5b9ff7b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:08.895088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2e1c2-6965-4e42-b0da-b77b5bb51422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:08.895807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2e1c2-6965-4e42-b0da-b77b5bb51422 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:08.896122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2e1c2-6965-4e42-b0da-b77b5bb51422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:08.897652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2e1c2-6965-4e42-b0da-b77b5bb51422 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:08.897652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2e1c2-6965-4e42-b0da-b77b5bb51422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:08.899091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2e1c2-6965-4e42-b0da-b77b5bb51422 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:09.033389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1208/2443] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:21:08 2022] POST /identity/v3/projects => generated 371 bytes in 539 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:09.038089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24dd5e-2a2f-4b99-9063-20fcf15133d4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:09.051970 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24dd5e-2a2f-4b99-9063-20fcf15133d4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:09.135066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56b45a-e50d-4000-b2da-51605514df4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:09.136026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56b45a-e50d-4000-b2da-51605514df4a admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:09.136495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56b45a-e50d-4000-b2da-51605514df4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:09.136939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56b45a-e50d-4000-b2da-51605514df4a admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:09.137981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56b45a-e50d-4000-b2da-51605514df4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:09.140516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56b45a-e50d-4000-b2da-51605514df4a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:09.277674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1236/2444] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:21:09 2022] POST /identity/v3/users => generated 292 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:09.282335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2e1c2-6965-4e42-b0da-b77b5bb51422 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:09.302316 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2e1c2-6965-4e42-b0da-b77b5bb51422 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:09.330635 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c29194-4af2-47a0-8316-cd00d98151e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:09.331663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c29194-4af2-47a0-8316-cd00d98151e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:09.336111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c29194-4af2-47a0-8316-cd00d98151e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:09.336633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c29194-4af2-47a0-8316-cd00d98151e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:09.338005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c29194-4af2-47a0-8316-cd00d98151e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:09.346587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c29194-4af2-47a0-8316-cd00d98151e3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:09.400081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56b45a-e50d-4000-b2da-51605514df4a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:09.400926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1209/2445] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:09.470011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56b45a-e50d-4000-b2da-51605514df4a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:09.484806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3daecd-52fa-48a3-ad0a-fd986a0c0f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:09.485677 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3daecd-52fa-48a3-ad0a-fd986a0c0f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:09.486013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3daecd-52fa-48a3-ad0a-fd986a0c0f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:09.486294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3daecd-52fa-48a3-ad0a-fd986a0c0f16 admin admin] PATH_INFO: `/v3/projects/56498dbba20e41f5a5b6b04e6d21a39b/users/d6ab2a6d480c4de1adb0de26132d1ba3/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:09.541662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3daecd-52fa-48a3-ad0a-fd986a0c0f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56498dbba20e41f5a5b6b04e6d21a39b, user_id=d6ab2a6d480c4de1adb0de26132d1ba3, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:09.544775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3daecd-52fa-48a3-ad0a-fd986a0c0f16 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:09.678502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1237/2446] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:09 2022] PUT /identity/v3/projects/56498dbba20e41f5a5b6b04e6d21a39b/users/d6ab2a6d480c4de1adb0de26132d1ba3/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:09.682660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c29194-4af2-47a0-8316-cd00d98151e3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:09.711558 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c29194-4af2-47a0-8316-cd00d98151e3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:09.865442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:09.888405 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639c645-124c-4771-a9f2-7bdf6460b862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:09.889427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639c645-124c-4771-a9f2-7bdf6460b862 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:09.889701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639c645-124c-4771-a9f2-7bdf6460b862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:09.890183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639c645-124c-4771-a9f2-7bdf6460b862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:09.891067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639c645-124c-4771-a9f2-7bdf6460b862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:09.905111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639c645-124c-4771-a9f2-7bdf6460b862 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:09.949582 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1210/2447] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:21:09.954098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3daecd-52fa-48a3-ad0a-fd986a0c0f16 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:09.973363 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3daecd-52fa-48a3-ad0a-fd986a0c0f16 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:09.991510 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bcece-3bc0-404b-9553-2f454dd2a952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:09.992914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bcece-3bc0-404b-9553-2f454dd2a952 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:09.993932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bcece-3bc0-404b-9553-2f454dd2a952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:09.993932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bcece-3bc0-404b-9553-2f454dd2a952 admin admin] PATH_INFO: `/v3/projects/56498dbba20e41f5a5b6b04e6d21a39b/users/d6ab2a6d480c4de1adb0de26132d1ba3/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:09.998586 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bcece-3bc0-404b-9553-2f454dd2a952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56498dbba20e41f5a5b6b04e6d21a39b, user_id=d6ab2a6d480c4de1adb0de26132d1ba3, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:10.005216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bcece-3bc0-404b-9553-2f454dd2a952 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:10.040533 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1238/2448] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:09 2022] PUT /identity/v3/projects/56498dbba20e41f5a5b6b04e6d21a39b/users/d6ab2a6d480c4de1adb0de26132d1ba3/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:11.823896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639c645-124c-4771-a9f2-7bdf6460b862 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:11.837305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639c645-124c-4771-a9f2-7bdf6460b862 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:12.024010 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75377d6-3796-433e-b271-0827c0e4facd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:12.026134 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75377d6-3796-433e-b271-0827c0e4facd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:12.026134 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75377d6-3796-433e-b271-0827c0e4facd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:12.026561 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75377d6-3796-433e-b271-0827c0e4facd admin admin] PATH_INFO: `/v3/users/66f0325855124a8b89fbe13aa13dbf9c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:12.029715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75377d6-3796-433e-b271-0827c0e4facd admin admin] RBAC: Authorizing `identity:delete_user(user_id=66f0325855124a8b89fbe13aa13dbf9c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:12.032153 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75377d6-3796-433e-b271-0827c0e4facd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:12.356918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d75377d6-3796-433e-b271-0827c0e4facd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66f0325855124a8b89fbe13aa13dbf9c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:12.433657 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d75377d6-3796-433e-b271-0827c0e4facd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66f0325855124a8b89fbe13aa13dbf9c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:12.442348 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d75377d6-3796-433e-b271-0827c0e4facd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66f0325855124a8b89fbe13aa13dbf9c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:12.466915 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d75377d6-3796-433e-b271-0827c0e4facd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66f0325855124a8b89fbe13aa13dbf9c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:12.480590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d75377d6-3796-433e-b271-0827c0e4facd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66f0325855124a8b89fbe13aa13dbf9c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:12.794292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1211/2449] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:21:11 2022] DELETE /identity/v3/users/66f0325855124a8b89fbe13aa13dbf9c => generated 0 bytes in 971 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:12.799223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ed8ca-4e3e-4b73-8dfd-4bb20946a81d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:12.799894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ed8ca-4e3e-4b73-8dfd-4bb20946a81d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:12.799894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ed8ca-4e3e-4b73-8dfd-4bb20946a81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:12.853593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a37ed8ca-4e3e-4b73-8dfd-4bb20946a81d None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:12.854930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a37ed8ca-4e3e-4b73-8dfd-4bb20946a81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:13.078766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1239/2450] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:21:13.091708 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75377d6-3796-433e-b271-0827c0e4facd admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:13.093779 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d75377d6-3796-433e-b271-0827c0e4facd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:13.323040 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:13.359702 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75377d6-3796-433e-b271-0827c0e4facd admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:13.374896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19a902-8173-4538-9590-e6f352373073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:13.374896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19a902-8173-4538-9590-e6f352373073 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:13.374896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19a902-8173-4538-9590-e6f352373073 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:13.374896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19a902-8173-4538-9590-e6f352373073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:13.398259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19a902-8173-4538-9590-e6f352373073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:13.400034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19a902-8173-4538-9590-e6f352373073 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:13.434746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1212/2451] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:21:14.349276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ed8ca-4e3e-4b73-8dfd-4bb20946a81d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:14.349276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a37ed8ca-4e3e-4b73-8dfd-4bb20946a81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:14.410237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ed8ca-4e3e-4b73-8dfd-4bb20946a81d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:14.430127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:14.435979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:14.436272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:14.436558 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c admin admin] PATH_INFO: `/v3/projects/fd17b9ad47244c20a6f61213ce1fb5ce` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:14.444636 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd17b9ad47244c20a6f61213ce1fb5ce)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:14.461738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:14.520856 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:14.737806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd17b9ad47244c20a6f61213ce1fb5ce', 'request_id': 'req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:14.754308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd17b9ad47244c20a6f61213ce1fb5ce', 'request_id': 'req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:14.866033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd17b9ad47244c20a6f61213ce1fb5ce', 'request_id': 'req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:14.904222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c admin admin] The token cache is being invalidate because project fd17b9ad47244c20a6f61213ce1fb5ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:21:14.904500 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:14.905157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:14.921064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1240/2452] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:21:14 2022] DELETE /identity/v3/projects/fd17b9ad47244c20a6f61213ce1fb5ce => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:18.654490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19a902-8173-4538-9590-e6f352373073 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:18.656701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fb19a902-8173-4538-9590-e6f352373073 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:18.696738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19a902-8173-4538-9590-e6f352373073 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:18.808269 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea1a03-2003-4ec4-bd5e-9c93da386e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:18.809460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea1a03-2003-4ec4-bd5e-9c93da386e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:18.809922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea1a03-2003-4ec4-bd5e-9c93da386e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:18.810351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea1a03-2003-4ec4-bd5e-9c93da386e97 admin admin] PATH_INFO: `/v3/users/b068a0e72d9e409aac407e3821649fb9` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:18.812004 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea1a03-2003-4ec4-bd5e-9c93da386e97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b068a0e72d9e409aac407e3821649fb9)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:18.814289 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea1a03-2003-4ec4-bd5e-9c93da386e97 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:19.331392 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d1ea1a03-2003-4ec4-bd5e-9c93da386e97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b068a0e72d9e409aac407e3821649fb9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:19.381380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d1ea1a03-2003-4ec4-bd5e-9c93da386e97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b068a0e72d9e409aac407e3821649fb9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:19.382769 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d1ea1a03-2003-4ec4-bd5e-9c93da386e97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b068a0e72d9e409aac407e3821649fb9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:19.409562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d1ea1a03-2003-4ec4-bd5e-9c93da386e97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b068a0e72d9e409aac407e3821649fb9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:19.411306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d1ea1a03-2003-4ec4-bd5e-9c93da386e97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b068a0e72d9e409aac407e3821649fb9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:19.517371 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1213/2453] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:21:18 2022] DELETE /identity/v3/users/b068a0e72d9e409aac407e3821649fb9 => generated 0 bytes in 859 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:19.943765 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:19.946650 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:19.981489 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83ce2c-6ebf-44eb-a152-0a2cae328b1c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:20.151669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c55532-01ec-48b7-9b81-53f978afe785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:20.152197 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c55532-01ec-48b7-9b81-53f978afe785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:20.152478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c55532-01ec-48b7-9b81-53f978afe785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:20.153382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c55532-01ec-48b7-9b81-53f978afe785 admin admin] PATH_INFO: `/v3/projects/a7676b5fcbc04076b2e9d0e36bbb07f1` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:20.154215 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c55532-01ec-48b7-9b81-53f978afe785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7676b5fcbc04076b2e9d0e36bbb07f1)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:20.156150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c55532-01ec-48b7-9b81-53f978afe785 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:20.324253 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f1c55532-01ec-48b7-9b81-53f978afe785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7676b5fcbc04076b2e9d0e36bbb07f1', 'request_id': 'req-f1c55532-01ec-48b7-9b81-53f978afe785'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:20.340146 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f1c55532-01ec-48b7-9b81-53f978afe785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7676b5fcbc04076b2e9d0e36bbb07f1', 'request_id': 'req-f1c55532-01ec-48b7-9b81-53f978afe785'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:20.406850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f1c55532-01ec-48b7-9b81-53f978afe785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7676b5fcbc04076b2e9d0e36bbb07f1', 'request_id': 'req-f1c55532-01ec-48b7-9b81-53f978afe785'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:20.424753 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f1c55532-01ec-48b7-9b81-53f978afe785 admin admin] The token cache is being invalidate because project a7676b5fcbc04076b2e9d0e36bbb07f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:21:20.425138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f1c55532-01ec-48b7-9b81-53f978afe785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:20.427542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f1c55532-01ec-48b7-9b81-53f978afe785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:20.433114 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1241/2454] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:21:19 2022] DELETE /identity/v3/projects/a7676b5fcbc04076b2e9d0e36bbb07f1 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:20.434970 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea1a03-2003-4ec4-bd5e-9c93da386e97 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:20.439847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d1ea1a03-2003-4ec4-bd5e-9c93da386e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:20.479270 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea1a03-2003-4ec4-bd5e-9c93da386e97 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:20.650995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774f2c6-0f90-44f1-8d19-5265dff6e5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:20.652446 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774f2c6-0f90-44f1-8d19-5265dff6e5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:20.652736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774f2c6-0f90-44f1-8d19-5265dff6e5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:20.653262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774f2c6-0f90-44f1-8d19-5265dff6e5d8 admin admin] PATH_INFO: `/v3/users/93fc52df31f446bcb7bfae01833adfc6` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:20.655435 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774f2c6-0f90-44f1-8d19-5265dff6e5d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93fc52df31f446bcb7bfae01833adfc6)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:20.661663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774f2c6-0f90-44f1-8d19-5265dff6e5d8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:20.667842 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:21.004393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2774f2c6-0f90-44f1-8d19-5265dff6e5d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93fc52df31f446bcb7bfae01833adfc6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:21.030143 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2774f2c6-0f90-44f1-8d19-5265dff6e5d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93fc52df31f446bcb7bfae01833adfc6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:21.030905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2774f2c6-0f90-44f1-8d19-5265dff6e5d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93fc52df31f446bcb7bfae01833adfc6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:21.052277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2774f2c6-0f90-44f1-8d19-5265dff6e5d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93fc52df31f446bcb7bfae01833adfc6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:21.053145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2774f2c6-0f90-44f1-8d19-5265dff6e5d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93fc52df31f446bcb7bfae01833adfc6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:21.183961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1214/2455] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:21:20 2022] DELETE /identity/v3/users/93fc52df31f446bcb7bfae01833adfc6 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:21.839965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c55532-01ec-48b7-9b81-53f978afe785 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:21.843051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f1c55532-01ec-48b7-9b81-53f978afe785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:21.860550 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c55532-01ec-48b7-9b81-53f978afe785 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:21.904154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:21.995305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693219a8-342d-4322-997d-97457a46beec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:21.999144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693219a8-342d-4322-997d-97457a46beec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:21.999144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693219a8-342d-4322-997d-97457a46beec admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:21.999144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693219a8-342d-4322-997d-97457a46beec admin admin] PATH_INFO: `/v3/projects/91e890bdd44142efabbe316506ef1a57` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:21.999144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693219a8-342d-4322-997d-97457a46beec admin admin] RBAC: Authorizing `identity:delete_project(project_id=91e890bdd44142efabbe316506ef1a57)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:22.001232 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693219a8-342d-4322-997d-97457a46beec admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:22.191867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-693219a8-342d-4322-997d-97457a46beec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e890bdd44142efabbe316506ef1a57', 'request_id': 'req-693219a8-342d-4322-997d-97457a46beec'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:22.200577 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-693219a8-342d-4322-997d-97457a46beec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e890bdd44142efabbe316506ef1a57', 'request_id': 'req-693219a8-342d-4322-997d-97457a46beec'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:22.305316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-693219a8-342d-4322-997d-97457a46beec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e890bdd44142efabbe316506ef1a57', 'request_id': 'req-693219a8-342d-4322-997d-97457a46beec'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:22.325887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-693219a8-342d-4322-997d-97457a46beec admin admin] The token cache is being invalidate because project 91e890bdd44142efabbe316506ef1a57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:21:22.326179 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-693219a8-342d-4322-997d-97457a46beec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:22.327073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-693219a8-342d-4322-997d-97457a46beec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:22.333450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1242/2456] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:21:21 2022] DELETE /identity/v3/projects/91e890bdd44142efabbe316506ef1a57 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:22.334699 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08716c7-add8-4534-9b70-c02adb83e678 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:22.335160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08716c7-add8-4534-9b70-c02adb83e678 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:22.335378 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08716c7-add8-4534-9b70-c02adb83e678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:22.427456 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e08716c7-add8-4534-9b70-c02adb83e678 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:22.430201 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e08716c7-add8-4534-9b70-c02adb83e678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:22.552576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1215/2457] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:22.560422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44a646-8fbd-4cb8-9499-5d38d099b781 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:22.560712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44a646-8fbd-4cb8-9499-5d38d099b781 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:22.560986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44a646-8fbd-4cb8-9499-5d38d099b781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:22.600729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ec44a646-8fbd-4cb8-9499-5d38d099b781 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:22.601601 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ec44a646-8fbd-4cb8-9499-5d38d099b781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:22.618242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1243/2458] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:22.622535 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08716c7-add8-4534-9b70-c02adb83e678 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:22.638133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08716c7-add8-4534-9b70-c02adb83e678 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:22.664654 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9c473-00a5-495b-a7b3-57702ac17a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:22.665724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9c473-00a5-495b-a7b3-57702ac17a38 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:22.666177 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9c473-00a5-495b-a7b3-57702ac17a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:22.666619 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9c473-00a5-495b-a7b3-57702ac17a38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:22.667599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9c473-00a5-495b-a7b3-57702ac17a38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:21:22.668086 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9c473-00a5-495b-a7b3-57702ac17a38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:22.669956 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9c473-00a5-495b-a7b3-57702ac17a38 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:22.692286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1216/2459] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:21:22.696462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44a646-8fbd-4cb8-9499-5d38d099b781 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:22.711773 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44a646-8fbd-4cb8-9499-5d38d099b781 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:22.739815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971cf22-f882-4c6f-9b59-db0a060b609b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:22.742419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971cf22-f882-4c6f-9b59-db0a060b609b admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:22.742419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971cf22-f882-4c6f-9b59-db0a060b609b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:22.742419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971cf22-f882-4c6f-9b59-db0a060b609b admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:22.742419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971cf22-f882-4c6f-9b59-db0a060b609b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:22.743847 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971cf22-f882-4c6f-9b59-db0a060b609b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:22.873471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1244/2460] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:21:22 2022] POST /identity/v3/projects => generated 359 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:22.877945 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9c473-00a5-495b-a7b3-57702ac17a38 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:22.888998 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9c473-00a5-495b-a7b3-57702ac17a38 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:22.982274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:23.025260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f22ae2-8cc7-42d5-8e8a-f64e0d3c5f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:23.025780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f22ae2-8cc7-42d5-8e8a-f64e0d3c5f48 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:23.026060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f22ae2-8cc7-42d5-8e8a-f64e0d3c5f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:23.026335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f22ae2-8cc7-42d5-8e8a-f64e0d3c5f48 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:23.027033 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f22ae2-8cc7-42d5-8e8a-f64e0d3c5f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:23.029274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f22ae2-8cc7-42d5-8e8a-f64e0d3c5f48 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:23.167416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971cf22-f882-4c6f-9b59-db0a060b609b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:23.182973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1217/2461] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:21:22 2022] POST /identity/v3/users => generated 280 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:23.191469 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971cf22-f882-4c6f-9b59-db0a060b609b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:23.209218 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f0427-9035-491a-bb53-e9108639d017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:23.211739 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f0427-9035-491a-bb53-e9108639d017 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:23.214525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f0427-9035-491a-bb53-e9108639d017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:23.214851 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f0427-9035-491a-bb53-e9108639d017 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:23.215647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f0427-9035-491a-bb53-e9108639d017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:23.227612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f0427-9035-491a-bb53-e9108639d017 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:23.305402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1245/2462] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:23.313607 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f22ae2-8cc7-42d5-8e8a-f64e0d3c5f48 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:23.325694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f22ae2-8cc7-42d5-8e8a-f64e0d3c5f48 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:23.351119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b3418-6fcb-4fb9-a287-6c9d673bc998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:23.351823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b3418-6fcb-4fb9-a287-6c9d673bc998 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:23.352093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b3418-6fcb-4fb9-a287-6c9d673bc998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:23.352344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b3418-6fcb-4fb9-a287-6c9d673bc998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:23.353092 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b3418-6fcb-4fb9-a287-6c9d673bc998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:23.354868 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b3418-6fcb-4fb9-a287-6c9d673bc998 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:23.399650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1218/2463] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:23.404010 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f0427-9035-491a-bb53-e9108639d017 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:23.434461 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f0427-9035-491a-bb53-e9108639d017 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:23.453673 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6054dc5-13c7-4f99-94ea-3b0ae96775a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:23.455237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6054dc5-13c7-4f99-94ea-3b0ae96775a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:23.455462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6054dc5-13c7-4f99-94ea-3b0ae96775a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:23.455860 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6054dc5-13c7-4f99-94ea-3b0ae96775a1 admin admin] PATH_INFO: `/v3/projects/d7ba7609809b47b29942396fe339f512/users/b8bb605db30b4b4fb89e5488fbe2d55a/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:23.481966 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6054dc5-13c7-4f99-94ea-3b0ae96775a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ba7609809b47b29942396fe339f512, user_id=b8bb605db30b4b4fb89e5488fbe2d55a, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:23.485668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6054dc5-13c7-4f99-94ea-3b0ae96775a1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:23.665727 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1246/2464] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:23 2022] PUT /identity/v3/projects/d7ba7609809b47b29942396fe339f512/users/b8bb605db30b4b4fb89e5488fbe2d55a/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:21:23.679047 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f90f8-b426-46d7-a886-0256ef2382be None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:23.679682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f90f8-b426-46d7-a886-0256ef2382be None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:23.680074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f90f8-b426-46d7-a886-0256ef2382be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:23.853807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-942f90f8-b426-46d7-a886-0256ef2382be None None] MFA Rules not processed for user `b8bb605db30b4b4fb89e5488fbe2d55a`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:23.853807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-942f90f8-b426-46d7-a886-0256ef2382be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:24.142236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1219/2465] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:23 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:24.177378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6054dc5-13c7-4f99-94ea-3b0ae96775a1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:24.179599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a6054dc5-13c7-4f99-94ea-3b0ae96775a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:24.254335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6054dc5-13c7-4f99-94ea-3b0ae96775a1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:24.258839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ed0e1-9019-43a9-ab88-adcdaa6316e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:24.259746 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ed0e1-9019-43a9-ab88-adcdaa6316e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:24.260279 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ed0e1-9019-43a9-ab88-adcdaa6316e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:24.260779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ed0e1-9019-43a9-ab88-adcdaa6316e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:24.273440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:24.277705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ed0e1-9019-43a9-ab88-adcdaa6316e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:24.283040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ed0e1-9019-43a9-ab88-adcdaa6316e0 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:24.297346 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1247/2466] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:24 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:21:24.315920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f90f8-b426-46d7-a886-0256ef2382be None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:24.485535 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f90f8-b426-46d7-a886-0256ef2382be None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:24.498625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef49850-b239-4f86-a830-f497a8b04abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:24.499720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef49850-b239-4f86-a830-f497a8b04abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:24.505352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef49850-b239-4f86-a830-f497a8b04abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:24.505352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef49850-b239-4f86-a830-f497a8b04abc admin admin] PATH_INFO: `/v3/users/b8bb605db30b4b4fb89e5488fbe2d55a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:24.505352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef49850-b239-4f86-a830-f497a8b04abc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8bb605db30b4b4fb89e5488fbe2d55a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:24.505352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef49850-b239-4f86-a830-f497a8b04abc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:24.730286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-eef49850-b239-4f86-a830-f497a8b04abc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8bb605db30b4b4fb89e5488fbe2d55a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:24.746728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:24.764433 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-eef49850-b239-4f86-a830-f497a8b04abc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8bb605db30b4b4fb89e5488fbe2d55a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:24.765536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-eef49850-b239-4f86-a830-f497a8b04abc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8bb605db30b4b4fb89e5488fbe2d55a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:24.775925 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-eef49850-b239-4f86-a830-f497a8b04abc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8bb605db30b4b4fb89e5488fbe2d55a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:24.776737 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-eef49850-b239-4f86-a830-f497a8b04abc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8bb605db30b4b4fb89e5488fbe2d55a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:24.873460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1220/2467] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:21:24 2022] DELETE /identity/v3/users/b8bb605db30b4b4fb89e5488fbe2d55a => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:21:24.876527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ef958-4292-4470-9502-0d955593e2c8 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:24.877513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ef958-4292-4470-9502-0d955593e2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:24.877937 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ef958-4292-4470-9502-0d955593e2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:24.934506 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-577ef958-4292-4470-9502-0d955593e2c8 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:24.935796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-577ef958-4292-4470-9502-0d955593e2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:25.112205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1248/2468] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:25.121703 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef49850-b239-4f86-a830-f497a8b04abc admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:25.123311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-eef49850-b239-4f86-a830-f497a8b04abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:25.160958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdbad5-c7c7-44b9-b678-59591e9e7d68 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:25.161723 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdbad5-c7c7-44b9-b678-59591e9e7d68 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:25.162376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdbad5-c7c7-44b9-b678-59591e9e7d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:25.201871 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c0bdbad5-c7c7-44b9-b678-59591e9e7d68 None None] MFA Rules not processed for user `d6ab2a6d480c4de1adb0de26132d1ba3`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:25.201871 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c0bdbad5-c7c7-44b9-b678-59591e9e7d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:25.264102 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef49850-b239-4f86-a830-f497a8b04abc admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:25.275254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b174c2-2934-48c8-953a-cfd44b9c5c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:25.275801 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b174c2-2934-48c8-953a-cfd44b9c5c05 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:25.276156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b174c2-2934-48c8-953a-cfd44b9c5c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:25.276438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b174c2-2934-48c8-953a-cfd44b9c5c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:25.296016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b174c2-2934-48c8-953a-cfd44b9c5c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:25.297583 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b174c2-2934-48c8-953a-cfd44b9c5c05 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:25.311559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1221/2469] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:21:25.331196 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1249/2470] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:25 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:25.346523 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b174c2-2934-48c8-953a-cfd44b9c5c05 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:25.350874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e7b174c2-2934-48c8-953a-cfd44b9c5c05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:25.567150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b174c2-2934-48c8-953a-cfd44b9c5c05 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:25.573109 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1d5e8-37e9-4ea2-a9ee-478e0311aeac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:25.573849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1d5e8-37e9-4ea2-a9ee-478e0311aeac service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:25.574202 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1d5e8-37e9-4ea2-a9ee-478e0311aeac service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:25.574544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1d5e8-37e9-4ea2-a9ee-478e0311aeac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:25.576449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a7c1d5e8-37e9-4ea2-a9ee-478e0311aeac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:25.623064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:25.640274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1d5e8-37e9-4ea2-a9ee-478e0311aeac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:25.641893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1d5e8-37e9-4ea2-a9ee-478e0311aeac service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:25.663203 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1222/2471] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:21:26.171434 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bdbad5-c7c7-44b9-b678-59591e9e7d68 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:26.172058 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c0bdbad5-c7c7-44b9-b678-59591e9e7d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:26.186179 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bdbad5-c7c7-44b9-b678-59591e9e7d68 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:26.197292 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efdba2-f048-4168-a43c-8dc3d1b0646e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:26.198142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efdba2-f048-4168-a43c-8dc3d1b0646e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:26.198533 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efdba2-f048-4168-a43c-8dc3d1b0646e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:26.198819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efdba2-f048-4168-a43c-8dc3d1b0646e admin admin] PATH_INFO: `/v3/projects/d7ba7609809b47b29942396fe339f512` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:26.200248 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efdba2-f048-4168-a43c-8dc3d1b0646e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7ba7609809b47b29942396fe339f512)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:26.202313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efdba2-f048-4168-a43c-8dc3d1b0646e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:26.328104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-17efdba2-f048-4168-a43c-8dc3d1b0646e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7ba7609809b47b29942396fe339f512', 'request_id': 'req-17efdba2-f048-4168-a43c-8dc3d1b0646e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:26.337837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-17efdba2-f048-4168-a43c-8dc3d1b0646e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7ba7609809b47b29942396fe339f512', 'request_id': 'req-17efdba2-f048-4168-a43c-8dc3d1b0646e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:26.402613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-17efdba2-f048-4168-a43c-8dc3d1b0646e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7ba7609809b47b29942396fe339f512', 'request_id': 'req-17efdba2-f048-4168-a43c-8dc3d1b0646e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:26.418041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-17efdba2-f048-4168-a43c-8dc3d1b0646e admin admin] The token cache is being invalidate because project d7ba7609809b47b29942396fe339f512 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:21:26.418550 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-17efdba2-f048-4168-a43c-8dc3d1b0646e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:26.420379 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-17efdba2-f048-4168-a43c-8dc3d1b0646e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:26.427389 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1250/2472] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:21:26 2022] DELETE /identity/v3/projects/d7ba7609809b47b29942396fe339f512 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:21:26.433185 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffd888-8f0f-4fdd-9951-754521fd42a5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:26.433690 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffd888-8f0f-4fdd-9951-754521fd42a5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:26.433977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffd888-8f0f-4fdd-9951-754521fd42a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:26.461356 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6cffd888-8f0f-4fdd-9951-754521fd42a5 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:26.463171 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6cffd888-8f0f-4fdd-9951-754521fd42a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:26.553698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1223/2473] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:21:26.557038 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08dbed7-fb5a-4218-874a-591ffb0051d2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:26.557406 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08dbed7-fb5a-4218-874a-591ffb0051d2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:26.557884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08dbed7-fb5a-4218-874a-591ffb0051d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:26.585667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07dd09-b83a-4a97-be89-5d7fc3e35d74 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:26.586138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07dd09-b83a-4a97-be89-5d7fc3e35d74 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:26.587557 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:26.588407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07dd09-b83a-4a97-be89-5d7fc3e35d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:26.621067 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a08dbed7-fb5a-4218-874a-591ffb0051d2 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:26.632419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a08dbed7-fb5a-4218-874a-591ffb0051d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:26.638751 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ff07dd09-b83a-4a97-be89-5d7fc3e35d74 None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:26.643596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ff07dd09-b83a-4a97-be89-5d7fc3e35d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:26.649435 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1251/2474] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:26.650470 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08dbed7-fb5a-4218-874a-591ffb0051d2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:26.664615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08dbed7-fb5a-4218-874a-591ffb0051d2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:26.670505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315cab0-1aab-408c-b9d7-96ab8d3c09ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:26.671336 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315cab0-1aab-408c-b9d7-96ab8d3c09ac admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:26.671953 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315cab0-1aab-408c-b9d7-96ab8d3c09ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:26.672311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315cab0-1aab-408c-b9d7-96ab8d3c09ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:26.674189 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315cab0-1aab-408c-b9d7-96ab8d3c09ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:21:26.674973 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315cab0-1aab-408c-b9d7-96ab8d3c09ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:26.678986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315cab0-1aab-408c-b9d7-96ab8d3c09ac admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:26.698783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1252/2475] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:21:26.700189 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315cab0-1aab-408c-b9d7-96ab8d3c09ac admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:26.714614 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315cab0-1aab-408c-b9d7-96ab8d3c09ac admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:26.720794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55acac-b0ef-467c-8867-f1a9c6676936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:26.721564 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55acac-b0ef-467c-8867-f1a9c6676936 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:26.721930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55acac-b0ef-467c-8867-f1a9c6676936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:26.722443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55acac-b0ef-467c-8867-f1a9c6676936 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:26.724201 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55acac-b0ef-467c-8867-f1a9c6676936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:26.729704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55acac-b0ef-467c-8867-f1a9c6676936 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:26.777253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1224/2476] 10.222.0.67 () {58 vars in 967 bytes} [Fri Jun 24 15:21:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:26.792106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07dd09-b83a-4a97-be89-5d7fc3e35d74 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:26.794203 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ff07dd09-b83a-4a97-be89-5d7fc3e35d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:26.835807 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1253/2477] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:21:26 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:26.841763 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55acac-b0ef-467c-8867-f1a9c6676936 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:26.921302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07dd09-b83a-4a97-be89-5d7fc3e35d74 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:26.954281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55acac-b0ef-467c-8867-f1a9c6676936 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:26.964676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22ba27-a78e-4b15-a028-788654262fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:26.965665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22ba27-a78e-4b15-a028-788654262fa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:26.965950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22ba27-a78e-4b15-a028-788654262fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:26.966780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22ba27-a78e-4b15-a028-788654262fa1 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:26.967560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22ba27-a78e-4b15-a028-788654262fa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:26.975456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22ba27-a78e-4b15-a028-788654262fa1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:26.975788 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:27.046224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa6cda-ffdf-4c58-b7dc-02d61e32ef95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:27.047388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa6cda-ffdf-4c58-b7dc-02d61e32ef95 service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:27.047899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa6cda-ffdf-4c58-b7dc-02d61e32ef95 service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:27.048908 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa6cda-ffdf-4c58-b7dc-02d61e32ef95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:27.071334 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1254/2478] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:21:26 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:27.074326 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22ba27-a78e-4b15-a028-788654262fa1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:27.085534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22ba27-a78e-4b15-a028-788654262fa1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:27.109728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bba6ab-20e3-40d1-8c46-2ca0ae68a832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:27.111133 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bba6ab-20e3-40d1-8c46-2ca0ae68a832 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:27.111878 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bba6ab-20e3-40d1-8c46-2ca0ae68a832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:27.113595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bba6ab-20e3-40d1-8c46-2ca0ae68a832 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:27.113595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bba6ab-20e3-40d1-8c46-2ca0ae68a832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:27.114975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bba6ab-20e3-40d1-8c46-2ca0ae68a832 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:27.134809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1255/2479] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:27.134809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bba6ab-20e3-40d1-8c46-2ca0ae68a832 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:27.148401 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa6cda-ffdf-4c58-b7dc-02d61e32ef95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:27.151408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa6cda-ffdf-4c58-b7dc-02d61e32ef95 service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:27.163579 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1225/2480] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:21:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 15:21:27.175033 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bba6ab-20e3-40d1-8c46-2ca0ae68a832 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:27.190395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963d7d6-fd82-4e13-9211-9524711a23c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:27.191906 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963d7d6-fd82-4e13-9211-9524711a23c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:27.192347 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963d7d6-fd82-4e13-9211-9524711a23c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:27.192778 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963d7d6-fd82-4e13-9211-9524711a23c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:27.193673 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963d7d6-fd82-4e13-9211-9524711a23c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:27.212417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963d7d6-fd82-4e13-9211-9524711a23c5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:27.225797 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1256/2481] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:27.233633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa6cda-ffdf-4c58-b7dc-02d61e32ef95 service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:27.259739 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa6cda-ffdf-4c58-b7dc-02d61e32ef95 service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:27.276793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc881ba3-fbaf-4eea-8bfe-fc0676ffe753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:27.279144 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc881ba3-fbaf-4eea-8bfe-fc0676ffe753 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:27.279722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc881ba3-fbaf-4eea-8bfe-fc0676ffe753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:27.280096 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc881ba3-fbaf-4eea-8bfe-fc0676ffe753 admin admin] PATH_INFO: `/v3/projects/47025fa9a1524b639d2940a0ab34e207/users/33088b427d134fdda32d1b7ed6630388/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:27.309182 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc881ba3-fbaf-4eea-8bfe-fc0676ffe753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47025fa9a1524b639d2940a0ab34e207, user_id=33088b427d134fdda32d1b7ed6630388, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:27.312234 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc881ba3-fbaf-4eea-8bfe-fc0676ffe753 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:27.455606 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1226/2482] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:27 2022] PUT /identity/v3/projects/47025fa9a1524b639d2940a0ab34e207/users/33088b427d134fdda32d1b7ed6630388/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:27.466773 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f65e0-38e4-494a-b9c5-faab3c660014 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:27.467083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f65e0-38e4-494a-b9c5-faab3c660014 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:27.467403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f65e0-38e4-494a-b9c5-faab3c660014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:27.517908 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ea6f65e0-38e4-494a-b9c5-faab3c660014 None None] MFA Rules not processed for user `33088b427d134fdda32d1b7ed6630388`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:27.521607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ea6f65e0-38e4-494a-b9c5-faab3c660014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:27.675344 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1257/2483] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:27.684010 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc881ba3-fbaf-4eea-8bfe-fc0676ffe753 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:27.686710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bc881ba3-fbaf-4eea-8bfe-fc0676ffe753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:27.776039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc881ba3-fbaf-4eea-8bfe-fc0676ffe753 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:27.780482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a4add-d810-4cb3-b611-2401e5e435d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:27.781156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4add-d810-4cb3-b611-2401e5e435d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:27.781525 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4add-d810-4cb3-b611-2401e5e435d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:27.781850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a4add-d810-4cb3-b611-2401e5e435d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:27.792937 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a4add-d810-4cb3-b611-2401e5e435d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:27.795087 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a4add-d810-4cb3-b611-2401e5e435d6 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:27.806960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1227/2484] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:21:29.466724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335d2fb-ee32-4851-a0e2-e14ba72e4a02 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:29.467210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335d2fb-ee32-4851-a0e2-e14ba72e4a02 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:29.467327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335d2fb-ee32-4851-a0e2-e14ba72e4a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:29.486270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9335d2fb-ee32-4851-a0e2-e14ba72e4a02 None None] MFA Rules not processed for user `bb55f38fa3c741cbb7f00e310368a85b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:29.487808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9335d2fb-ee32-4851-a0e2-e14ba72e4a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:29.502605 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:29.546856 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1258/2485] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:31.085193 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23600668-aca9-4771-897e-97eca0bbc2b1 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:31.086787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23600668-aca9-4771-897e-97eca0bbc2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:31.086787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23600668-aca9-4771-897e-97eca0bbc2b1 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:31.088374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1228/2486] 10.222.0.67 () {54 vars in 921 bytes} [Fri Jun 24 15:21:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:21:31.095601 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659c292-33f6-4ff2-a35b-890002c03281 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:31.096178 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659c292-33f6-4ff2-a35b-890002c03281 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:31.096178 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659c292-33f6-4ff2-a35b-890002c03281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:31.138341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6659c292-33f6-4ff2-a35b-890002c03281 None None] MFA Rules not processed for user `51de59caa3144a1cbf12b20b15e2134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:31.139631 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6659c292-33f6-4ff2-a35b-890002c03281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:31.232262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1259/2487] 10.222.0.67 () {58 vars in 1018 bytes} [Fri Jun 24 15:21:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:31.244760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23600668-aca9-4771-897e-97eca0bbc2b1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:31.246456 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-23600668-aca9-4771-897e-97eca0bbc2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:31.302347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:31.310083 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659c292-33f6-4ff2-a35b-890002c03281 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:31.310757 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6659c292-33f6-4ff2-a35b-890002c03281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:31.319292 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23600668-aca9-4771-897e-97eca0bbc2b1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:31.322299 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c8675-35b2-415b-a015-c4754a0a6dbe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1971c464293f426f97d6cb199e48193b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:31.322975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c8675-35b2-415b-a015-c4754a0a6dbe service glance] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:31.323319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c8675-35b2-415b-a015-c4754a0a6dbe service glance] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:31.323606 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c8675-35b2-415b-a015-c4754a0a6dbe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:31.339357 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c8675-35b2-415b-a015-c4754a0a6dbe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:31.339357 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c8675-35b2-415b-a015-c4754a0a6dbe service glance] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:31.350009 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1229/2488] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:21:31.400447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659c292-33f6-4ff2-a35b-890002c03281 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:31.405643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0d608-3fd4-409f-9de7-c2c3637f750a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:31.406724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0d608-3fd4-409f-9de7-c2c3637f750a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:31.407229 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0d608-3fd4-409f-9de7-c2c3637f750a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:31.408150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0d608-3fd4-409f-9de7-c2c3637f750a admin admin] PATH_INFO: `/v3/users/c95b2f1348424e6fa05c9599d4e3611a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:31.409392 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b0d608-3fd4-409f-9de7-c2c3637f750a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c95b2f1348424e6fa05c9599d4e3611a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:31.411564 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b0d608-3fd4-409f-9de7-c2c3637f750a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:31.581291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5cbcf-ffca-4fa6-9a3f-1f713224f043 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:31.581578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5cbcf-ffca-4fa6-9a3f-1f713224f043 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:31.581849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5cbcf-ffca-4fa6-9a3f-1f713224f043 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:31.584077 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1230/2489] 10.222.0.67 () {54 vars in 921 bytes} [Fri Jun 24 15:21:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:21:31.587888 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778ffff-f0e2-403f-accb-e7bccd07b2f0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:31.588014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:31.588295 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778ffff-f0e2-403f-accb-e7bccd07b2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:31.588578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778ffff-f0e2-403f-accb-e7bccd07b2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:31.589815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-26b0d608-3fd4-409f-9de7-c2c3637f750a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c95b2f1348424e6fa05c9599d4e3611a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:31.604583 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-26b0d608-3fd4-409f-9de7-c2c3637f750a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c95b2f1348424e6fa05c9599d4e3611a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:31.606898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8778ffff-f0e2-403f-accb-e7bccd07b2f0 None None] MFA Rules not processed for user `95e7fb8f48dd4348b54e3694a8410bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:31.608133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8778ffff-f0e2-403f-accb-e7bccd07b2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:31.659655 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1231/2490] 10.222.0.67 () {58 vars in 1018 bytes} [Fri Jun 24 15:21:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:21:31.671485 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778ffff-f0e2-403f-accb-e7bccd07b2f0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:31.675856 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8778ffff-f0e2-403f-accb-e7bccd07b2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:31.683118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778ffff-f0e2-403f-accb-e7bccd07b2f0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:31.686710 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38221de8-42df-4750-8e93-cde8c24311ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:31.687377 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38221de8-42df-4750-8e93-cde8c24311ba service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:31.687660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38221de8-42df-4750-8e93-cde8c24311ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:31.687938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38221de8-42df-4750-8e93-cde8c24311ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:31.693939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-26b0d608-3fd4-409f-9de7-c2c3637f750a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c95b2f1348424e6fa05c9599d4e3611a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:31.694386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-26b0d608-3fd4-409f-9de7-c2c3637f750a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c95b2f1348424e6fa05c9599d4e3611a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:31.695833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38221de8-42df-4750-8e93-cde8c24311ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:31.698890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38221de8-42df-4750-8e93-cde8c24311ba service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:31.702085 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-26b0d608-3fd4-409f-9de7-c2c3637f750a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c95b2f1348424e6fa05c9599d4e3611a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:31.705907 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1232/2491] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:21:31.713609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1260/2492] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:21:31 2022] DELETE /identity/v3/users/c95b2f1348424e6fa05c9599d4e3611a => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:31.806717 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b2167-dc76-42cc-8eb6-438488547f95 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:31.807017 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b2167-dc76-42cc-8eb6-438488547f95 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:31.807378 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b2167-dc76-42cc-8eb6-438488547f95 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:31.809280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1233/2493] 10.222.0.67 () {54 vars in 921 bytes} [Fri Jun 24 15:21:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 15:21:31.815820 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf602e9-3c17-4df6-864c-5ae4d08613ea None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:31.816108 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf602e9-3c17-4df6-864c-5ae4d08613ea None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:31.816531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf602e9-3c17-4df6-864c-5ae4d08613ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:31.836736 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-cbf602e9-3c17-4df6-864c-5ae4d08613ea None None] MFA Rules not processed for user `51de59caa3144a1cbf12b20b15e2134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:31.838013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cbf602e9-3c17-4df6-864c-5ae4d08613ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:31.911785 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1261/2494] 10.222.0.67 () {58 vars in 1018 bytes} [Fri Jun 24 15:21:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:31.922455 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b2167-dc76-42cc-8eb6-438488547f95 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:31.924759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0b9b2167-dc76-42cc-8eb6-438488547f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:31.990763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b2167-dc76-42cc-8eb6-438488547f95 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:31.994265 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cb76d-af58-47cd-8a67-f65b617f93d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:31.994922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cb76d-af58-47cd-8a67-f65b617f93d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:31.995227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cb76d-af58-47cd-8a67-f65b617f93d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:31.995511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cb76d-af58-47cd-8a67-f65b617f93d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:32.006170 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cb76d-af58-47cd-8a67-f65b617f93d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:32.007647 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cb76d-af58-47cd-8a67-f65b617f93d1 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:32.019395 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1234/2495] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:21:32.163444 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf602e9-3c17-4df6-864c-5ae4d08613ea None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:32.164980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cbf602e9-3c17-4df6-864c-5ae4d08613ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:32.242460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf602e9-3c17-4df6-864c-5ae4d08613ea None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:32.247283 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80705d1-1be2-418b-bf3f-de558ea3277d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:32.248161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80705d1-1be2-418b-bf3f-de558ea3277d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:32.248444 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80705d1-1be2-418b-bf3f-de558ea3277d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:32.248722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80705d1-1be2-418b-bf3f-de558ea3277d admin admin] PATH_INFO: `/v3/projects/b0fa019dc14d4e40a77d6b7db5cad9be` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:32.249924 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80705d1-1be2-418b-bf3f-de558ea3277d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0fa019dc14d4e40a77d6b7db5cad9be)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:32.252495 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80705d1-1be2-418b-bf3f-de558ea3277d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:32.380350 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d80705d1-1be2-418b-bf3f-de558ea3277d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0fa019dc14d4e40a77d6b7db5cad9be', 'request_id': 'req-d80705d1-1be2-418b-bf3f-de558ea3277d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:32.386931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d80705d1-1be2-418b-bf3f-de558ea3277d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0fa019dc14d4e40a77d6b7db5cad9be', 'request_id': 'req-d80705d1-1be2-418b-bf3f-de558ea3277d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:32.390255 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:32.485879 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d80705d1-1be2-418b-bf3f-de558ea3277d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0fa019dc14d4e40a77d6b7db5cad9be', 'request_id': 'req-d80705d1-1be2-418b-bf3f-de558ea3277d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:32.496760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d80705d1-1be2-418b-bf3f-de558ea3277d admin admin] The token cache is being invalidate because project b0fa019dc14d4e40a77d6b7db5cad9be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:21:32.497056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d80705d1-1be2-418b-bf3f-de558ea3277d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:32.497704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d80705d1-1be2-418b-bf3f-de558ea3277d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:32.499309 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1262/2496] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:21:32 2022] DELETE /identity/v3/projects/b0fa019dc14d4e40a77d6b7db5cad9be => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:32.504976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458ca1f-5213-4a93-91e4-b46164f806bb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:32.505454 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458ca1f-5213-4a93-91e4-b46164f806bb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:32.505553 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458ca1f-5213-4a93-91e4-b46164f806bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:32.524813 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8458ca1f-5213-4a93-91e4-b46164f806bb None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:32.526021 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8458ca1f-5213-4a93-91e4-b46164f806bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:32.586142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1235/2497] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:32.593814 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e81cd-47a2-4fdf-a17f-6cabc6efd595 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:32.594105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e81cd-47a2-4fdf-a17f-6cabc6efd595 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:32.594382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e81cd-47a2-4fdf-a17f-6cabc6efd595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:32.619112 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8e6e81cd-47a2-4fdf-a17f-6cabc6efd595 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:32.621494 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8e6e81cd-47a2-4fdf-a17f-6cabc6efd595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:32.633448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1263/2498] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:32.634759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458ca1f-5213-4a93-91e4-b46164f806bb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:32.645526 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458ca1f-5213-4a93-91e4-b46164f806bb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:32.650112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4608d3-b200-439c-987a-b61cdd900d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:32.653327 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4608d3-b200-439c-987a-b61cdd900d96 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:32.653327 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4608d3-b200-439c-987a-b61cdd900d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:32.653327 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4608d3-b200-439c-987a-b61cdd900d96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:32.653585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4608d3-b200-439c-987a-b61cdd900d96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:21:32.653776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4608d3-b200-439c-987a-b61cdd900d96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:32.655421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4608d3-b200-439c-987a-b61cdd900d96 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:32.667809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1236/2499] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:21:32.672204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e81cd-47a2-4fdf-a17f-6cabc6efd595 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:32.681893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e81cd-47a2-4fdf-a17f-6cabc6efd595 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:32.687933 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a4af4-1b14-4095-b52b-82c704b5073f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:32.693203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a4af4-1b14-4095-b52b-82c704b5073f admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:32.693203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a4af4-1b14-4095-b52b-82c704b5073f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:32.693203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a4af4-1b14-4095-b52b-82c704b5073f admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:32.693203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a4af4-1b14-4095-b52b-82c704b5073f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:32.698157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a4af4-1b14-4095-b52b-82c704b5073f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:32.791791 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1264/2500] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:21:32 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:21:32.795999 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4608d3-b200-439c-987a-b61cdd900d96 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:32.806587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4608d3-b200-439c-987a-b61cdd900d96 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:32.816098 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:32.861651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55be01-d231-4ad1-a480-8aba8fd21fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:32.862255 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55be01-d231-4ad1-a480-8aba8fd21fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:32.862542 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55be01-d231-4ad1-a480-8aba8fd21fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:32.862841 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55be01-d231-4ad1-a480-8aba8fd21fd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:32.863552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55be01-d231-4ad1-a480-8aba8fd21fd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:32.865504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55be01-d231-4ad1-a480-8aba8fd21fd5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:32.932295 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1237/2501] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:21:32 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:32.935877 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a4af4-1b14-4095-b52b-82c704b5073f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:32.946855 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a4af4-1b14-4095-b52b-82c704b5073f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:32.951706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f14046-8bf3-4b5e-a7bd-147896dda307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:32.952377 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f14046-8bf3-4b5e-a7bd-147896dda307 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:32.952662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f14046-8bf3-4b5e-a7bd-147896dda307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:32.952943 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f14046-8bf3-4b5e-a7bd-147896dda307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:32.953666 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f14046-8bf3-4b5e-a7bd-147896dda307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:32.955516 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f14046-8bf3-4b5e-a7bd-147896dda307 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:32.966368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1265/2502] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:32.970438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55be01-d231-4ad1-a480-8aba8fd21fd5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:32.980674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55be01-d231-4ad1-a480-8aba8fd21fd5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:32.985847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e28cf4-5282-48e4-9f5d-0c403ff41279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:32.986517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e28cf4-5282-48e4-9f5d-0c403ff41279 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:32.986797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e28cf4-5282-48e4-9f5d-0c403ff41279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:32.987075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e28cf4-5282-48e4-9f5d-0c403ff41279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:32.987845 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e28cf4-5282-48e4-9f5d-0c403ff41279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:32.989442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e28cf4-5282-48e4-9f5d-0c403ff41279 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:32.999428 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1238/2503] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:33.004378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f14046-8bf3-4b5e-a7bd-147896dda307 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:33.014173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f14046-8bf3-4b5e-a7bd-147896dda307 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:33.018994 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37fc6e-8e00-437e-9746-12edc7b6e76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:33.019905 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37fc6e-8e00-437e-9746-12edc7b6e76f admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:33.020191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37fc6e-8e00-437e-9746-12edc7b6e76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:33.020468 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37fc6e-8e00-437e-9746-12edc7b6e76f admin admin] PATH_INFO: `/v3/projects/140b17816f394e9cbc609a2e349311e0/users/1b56e012641d425ab395693aebed5836/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:33.039521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37fc6e-8e00-437e-9746-12edc7b6e76f admin admin] RBAC: Authorizing `identity:create_grant(project_id=140b17816f394e9cbc609a2e349311e0, user_id=1b56e012641d425ab395693aebed5836, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:33.042177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37fc6e-8e00-437e-9746-12edc7b6e76f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:33.151121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1266/2504] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:33 2022] PUT /identity/v3/projects/140b17816f394e9cbc609a2e349311e0/users/1b56e012641d425ab395693aebed5836/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:33.161657 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63878a9-c453-46e8-8ab1-4b731d9ba48c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:33.161951 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63878a9-c453-46e8-8ab1-4b731d9ba48c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:33.162233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63878a9-c453-46e8-8ab1-4b731d9ba48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:33.193158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f63878a9-c453-46e8-8ab1-4b731d9ba48c None None] MFA Rules not processed for user `1b56e012641d425ab395693aebed5836`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:33.194379 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f63878a9-c453-46e8-8ab1-4b731d9ba48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:33.262078 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1239/2505] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:33.273301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37fc6e-8e00-437e-9746-12edc7b6e76f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:33.274964 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4c37fc6e-8e00-437e-9746-12edc7b6e76f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:33.353787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37fc6e-8e00-437e-9746-12edc7b6e76f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:33.357528 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce2a59-a42b-422e-9a26-3e16ad90543c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:33.358167 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce2a59-a42b-422e-9a26-3e16ad90543c service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:33.358449 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce2a59-a42b-422e-9a26-3e16ad90543c service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:33.358725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce2a59-a42b-422e-9a26-3e16ad90543c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:33.367315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce2a59-a42b-422e-9a26-3e16ad90543c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:33.368730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce2a59-a42b-422e-9a26-3e16ad90543c service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:33.378460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1267/2506] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:21:39.760822 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63878a9-c453-46e8-8ab1-4b731d9ba48c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:39.827791 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63878a9-c453-46e8-8ab1-4b731d9ba48c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:39.832566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d998d24-48d3-41f6-b983-ece9bea7caf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:39.833441 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d998d24-48d3-41f6-b983-ece9bea7caf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:39.833723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d998d24-48d3-41f6-b983-ece9bea7caf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:39.833995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d998d24-48d3-41f6-b983-ece9bea7caf5 admin admin] PATH_INFO: `/v3/users/1b56e012641d425ab395693aebed5836` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:39.835213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d998d24-48d3-41f6-b983-ece9bea7caf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b56e012641d425ab395693aebed5836)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:39.837156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d998d24-48d3-41f6-b983-ece9bea7caf5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:39.862846 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:40.026303 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce2a59-a42b-422e-9a26-3e16ad90543c service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:40.028511 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3bce2a59-a42b-422e-9a26-3e16ad90543c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:40.078258 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2d998d24-48d3-41f6-b983-ece9bea7caf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b56e012641d425ab395693aebed5836'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:40.096638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2d998d24-48d3-41f6-b983-ece9bea7caf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b56e012641d425ab395693aebed5836'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:40.097908 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2d998d24-48d3-41f6-b983-ece9bea7caf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b56e012641d425ab395693aebed5836'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:40.105898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2d998d24-48d3-41f6-b983-ece9bea7caf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b56e012641d425ab395693aebed5836'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:40.106658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2d998d24-48d3-41f6-b983-ece9bea7caf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b56e012641d425ab395693aebed5836'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:40.147964 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:40.176743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce2a59-a42b-422e-9a26-3e16ad90543c service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:40.181723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1240/2507] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:21:39 2022] DELETE /identity/v3/users/1b56e012641d425ab395693aebed5836 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:40.184066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6265b1f-806e-453e-bf19-165669f6bede service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:40.184926 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6265b1f-806e-453e-bf19-165669f6bede service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:40.185241 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6265b1f-806e-453e-bf19-165669f6bede service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:40.185521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6265b1f-806e-453e-bf19-165669f6bede service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:40.186837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a460d2-f95d-463b-aed3-638eece92d7f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:40.187418 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a460d2-f95d-463b-aed3-638eece92d7f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:40.187824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d6265b1f-806e-453e-bf19-165669f6bede service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:40.188070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a460d2-f95d-463b-aed3-638eece92d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:40.204398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6265b1f-806e-453e-bf19-165669f6bede service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:40.206025 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6265b1f-806e-453e-bf19-165669f6bede service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:40.208754 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-69a460d2-f95d-463b-aed3-638eece92d7f None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:40.209992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-69a460d2-f95d-463b-aed3-638eece92d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:40.217812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1268/2508] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:21:40.283823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1241/2509] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:21:40.294038 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6265b1f-806e-453e-bf19-165669f6bede service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:40.295581 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d6265b1f-806e-453e-bf19-165669f6bede service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:40.357101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6265b1f-806e-453e-bf19-165669f6bede service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:40.421170 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ba262-7548-4b27-8d26-4ea7042c773e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:40.421815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ba262-7548-4b27-8d26-4ea7042c773e service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:40.422094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ba262-7548-4b27-8d26-4ea7042c773e service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:40.422366 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ba262-7548-4b27-8d26-4ea7042c773e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:40.431788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ba262-7548-4b27-8d26-4ea7042c773e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:40.433213 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ba262-7548-4b27-8d26-4ea7042c773e service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:40.443578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1269/2510] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:21:41.001543 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a460d2-f95d-463b-aed3-638eece92d7f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:41.004838 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-69a460d2-f95d-463b-aed3-638eece92d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:41.017554 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a460d2-f95d-463b-aed3-638eece92d7f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:41.026586 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35344d-9cd2-4b69-86f7-0cd7e0d19e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:41.027510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35344d-9cd2-4b69-86f7-0cd7e0d19e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:41.028564 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35344d-9cd2-4b69-86f7-0cd7e0d19e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:41.028564 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35344d-9cd2-4b69-86f7-0cd7e0d19e78 admin admin] PATH_INFO: `/v3/projects/140b17816f394e9cbc609a2e349311e0` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:41.029690 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35344d-9cd2-4b69-86f7-0cd7e0d19e78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=140b17816f394e9cbc609a2e349311e0)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:41.031724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35344d-9cd2-4b69-86f7-0cd7e0d19e78 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:41.183326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1d35344d-9cd2-4b69-86f7-0cd7e0d19e78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '140b17816f394e9cbc609a2e349311e0', 'request_id': 'req-1d35344d-9cd2-4b69-86f7-0cd7e0d19e78'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:41.193046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1d35344d-9cd2-4b69-86f7-0cd7e0d19e78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '140b17816f394e9cbc609a2e349311e0', 'request_id': 'req-1d35344d-9cd2-4b69-86f7-0cd7e0d19e78'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:41.293570 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1d35344d-9cd2-4b69-86f7-0cd7e0d19e78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '140b17816f394e9cbc609a2e349311e0', 'request_id': 'req-1d35344d-9cd2-4b69-86f7-0cd7e0d19e78'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:41.305516 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1d35344d-9cd2-4b69-86f7-0cd7e0d19e78 admin admin] The token cache is being invalidate because project 140b17816f394e9cbc609a2e349311e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:21:41.305808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1d35344d-9cd2-4b69-86f7-0cd7e0d19e78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:41.306442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1d35344d-9cd2-4b69-86f7-0cd7e0d19e78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:41.308049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1242/2511] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:21:41 2022] DELETE /identity/v3/projects/140b17816f394e9cbc609a2e349311e0 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:21:41.313998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f41ca-9ca8-41de-bf52-923a3a36e327 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:41.314655 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f41ca-9ca8-41de-bf52-923a3a36e327 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:41.314655 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f41ca-9ca8-41de-bf52-923a3a36e327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:41.333594 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-638f41ca-9ca8-41de-bf52-923a3a36e327 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:41.334810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-638f41ca-9ca8-41de-bf52-923a3a36e327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:41.403517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1270/2512] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:41.412901 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5fb15-d039-442b-90a5-92d014b85068 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:41.413209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5fb15-d039-442b-90a5-92d014b85068 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:41.413511 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5fb15-d039-442b-90a5-92d014b85068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:41.432505 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-fff5fb15-d039-442b-90a5-92d014b85068 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:41.433707 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fff5fb15-d039-442b-90a5-92d014b85068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:41.441816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1243/2513] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:41.445993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f41ca-9ca8-41de-bf52-923a3a36e327 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:41.456674 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f41ca-9ca8-41de-bf52-923a3a36e327 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:41.461670 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c68e8-475d-4773-9ef8-e567d8332bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:41.462346 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c68e8-475d-4773-9ef8-e567d8332bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:41.462639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c68e8-475d-4773-9ef8-e567d8332bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:41.462920 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c68e8-475d-4773-9ef8-e567d8332bd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:41.463699 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c68e8-475d-4773-9ef8-e567d8332bd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:21:41.464012 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c68e8-475d-4773-9ef8-e567d8332bd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:41.465656 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c68e8-475d-4773-9ef8-e567d8332bd9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:41.478244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1271/2514] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:21:41.482413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5fb15-d039-442b-90a5-92d014b85068 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:41.492295 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5fb15-d039-442b-90a5-92d014b85068 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:41.496904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f06a2-091b-46c7-95b7-5aa8b7ac0fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:41.497556 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f06a2-091b-46c7-95b7-5aa8b7ac0fed admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:41.497841 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f06a2-091b-46c7-95b7-5aa8b7ac0fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:41.498117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f06a2-091b-46c7-95b7-5aa8b7ac0fed admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:41.498804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f06a2-091b-46c7-95b7-5aa8b7ac0fed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:41.500739 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f06a2-091b-46c7-95b7-5aa8b7ac0fed admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:41.514436 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:41.602946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1244/2515] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:21:41 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:41.606613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c68e8-475d-4773-9ef8-e567d8332bd9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:41.618435 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c68e8-475d-4773-9ef8-e567d8332bd9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:41.635631 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:41.676683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe6d6a-6cbb-4fe6-8f44-9e608dc46c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:41.677367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe6d6a-6cbb-4fe6-8f44-9e608dc46c55 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:41.677658 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe6d6a-6cbb-4fe6-8f44-9e608dc46c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:41.677942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe6d6a-6cbb-4fe6-8f44-9e608dc46c55 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:41.678652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe6d6a-6cbb-4fe6-8f44-9e608dc46c55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:41.681206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe6d6a-6cbb-4fe6-8f44-9e608dc46c55 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:41.769952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1272/2516] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:21:41 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:41.785235 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f06a2-091b-46c7-95b7-5aa8b7ac0fed admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:41.796168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f06a2-091b-46c7-95b7-5aa8b7ac0fed admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:41.801571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bb7ab-f393-4bf3-b47c-55070bccdb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:41.802364 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bb7ab-f393-4bf3-b47c-55070bccdb23 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:41.802650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bb7ab-f393-4bf3-b47c-55070bccdb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:41.802924 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bb7ab-f393-4bf3-b47c-55070bccdb23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:41.803678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bb7ab-f393-4bf3-b47c-55070bccdb23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:41.806484 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bb7ab-f393-4bf3-b47c-55070bccdb23 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:41.823079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1245/2517] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:41.826940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe6d6a-6cbb-4fe6-8f44-9e608dc46c55 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:41.838964 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe6d6a-6cbb-4fe6-8f44-9e608dc46c55 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:41.844288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c544ff-6803-4345-9c60-8defaf285ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:41.851417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c544ff-6803-4345-9c60-8defaf285ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:41.851417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c544ff-6803-4345-9c60-8defaf285ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:41.851417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c544ff-6803-4345-9c60-8defaf285ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:41.851417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c544ff-6803-4345-9c60-8defaf285ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:41.851417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c544ff-6803-4345-9c60-8defaf285ce3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:41.868805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1273/2518] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:41.870942 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bb7ab-f393-4bf3-b47c-55070bccdb23 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:41.885585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bb7ab-f393-4bf3-b47c-55070bccdb23 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:41.888732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7ee0d-ae7e-48dd-ad14-8bc269a72e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:41.889605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7ee0d-ae7e-48dd-ad14-8bc269a72e1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:41.889885 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7ee0d-ae7e-48dd-ad14-8bc269a72e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:41.890162 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7ee0d-ae7e-48dd-ad14-8bc269a72e1e admin admin] PATH_INFO: `/v3/projects/fb2390854b0345588bd6b68f5c09a1ee/users/e4f136ae627b4a81b6b29c667420f1aa/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:41.905197 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7ee0d-ae7e-48dd-ad14-8bc269a72e1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb2390854b0345588bd6b68f5c09a1ee, user_id=e4f136ae627b4a81b6b29c667420f1aa, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:41.907853 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7ee0d-ae7e-48dd-ad14-8bc269a72e1e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:41.978749 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1246/2519] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:41 2022] PUT /identity/v3/projects/fb2390854b0345588bd6b68f5c09a1ee/users/e4f136ae627b4a81b6b29c667420f1aa/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:21:41.983805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00488583-ee26-421c-a0a4-afccecbffa7a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:41.983968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00488583-ee26-421c-a0a4-afccecbffa7a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:41.984306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00488583-ee26-421c-a0a4-afccecbffa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:42.006419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-00488583-ee26-421c-a0a4-afccecbffa7a None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:42.007981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-00488583-ee26-421c-a0a4-afccecbffa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:42.100855 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1274/2520] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:42.119884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7ee0d-ae7e-48dd-ad14-8bc269a72e1e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:42.119884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-cdd7ee0d-ae7e-48dd-ad14-8bc269a72e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:42.191787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7ee0d-ae7e-48dd-ad14-8bc269a72e1e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:42.202423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cf223-bf03-449d-b26f-4147da37e16a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:42.203897 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cf223-bf03-449d-b26f-4147da37e16a service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:42.204507 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cf223-bf03-449d-b26f-4147da37e16a service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:42.205186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cf223-bf03-449d-b26f-4147da37e16a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:42.225920 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43cf223-bf03-449d-b26f-4147da37e16a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:42.227501 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43cf223-bf03-449d-b26f-4147da37e16a service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:42.239704 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1247/2521] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:21:49.255743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592f11e-69ba-4c8f-8de4-2ee7a17a1b93 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:49.256275 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592f11e-69ba-4c8f-8de4-2ee7a17a1b93 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:49.256368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592f11e-69ba-4c8f-8de4-2ee7a17a1b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:49.283593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2592f11e-69ba-4c8f-8de4-2ee7a17a1b93 None None] MFA Rules not processed for user `e4f136ae627b4a81b6b29c667420f1aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:49.284787 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2592f11e-69ba-4c8f-8de4-2ee7a17a1b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:49.366616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1275/2522] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:49.371122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cf223-bf03-449d-b26f-4147da37e16a service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:49.382037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cf223-bf03-449d-b26f-4147da37e16a service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:49.386861 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2f303-78b8-47f5-850a-51ce7f2a47da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:49.387547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2f303-78b8-47f5-850a-51ce7f2a47da admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:49.387829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2f303-78b8-47f5-850a-51ce7f2a47da admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:49.388101 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2f303-78b8-47f5-850a-51ce7f2a47da admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:49.388798 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2f303-78b8-47f5-850a-51ce7f2a47da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:49.390816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2f303-78b8-47f5-850a-51ce7f2a47da admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:49.476173 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1248/2523] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:21:49 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:49.480388 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592f11e-69ba-4c8f-8de4-2ee7a17a1b93 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:49.491826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592f11e-69ba-4c8f-8de4-2ee7a17a1b93 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:49.526421 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:49.557872 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce78b3f-75cf-4c3b-a029-2d61e4e5fa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:49.558607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78b3f-75cf-4c3b-a029-2d61e4e5fa91 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:49.558866 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78b3f-75cf-4c3b-a029-2d61e4e5fa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:49.559140 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78b3f-75cf-4c3b-a029-2d61e4e5fa91 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:49.559882 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce78b3f-75cf-4c3b-a029-2d61e4e5fa91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:49.561942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce78b3f-75cf-4c3b-a029-2d61e4e5fa91 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:49.689912 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1276/2524] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:21:49 2022] POST /identity/v3/users => generated 298 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:49.691943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2f303-78b8-47f5-850a-51ce7f2a47da admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:49.702310 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2f303-78b8-47f5-850a-51ce7f2a47da admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:49.707908 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00d746-eb9a-4012-8422-89fc1fa74857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:49.708683 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00d746-eb9a-4012-8422-89fc1fa74857 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:49.709070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00d746-eb9a-4012-8422-89fc1fa74857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:49.709432 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00d746-eb9a-4012-8422-89fc1fa74857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:49.710409 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00d746-eb9a-4012-8422-89fc1fa74857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:49.714317 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00d746-eb9a-4012-8422-89fc1fa74857 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:49.725904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1249/2525] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:49.730103 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce78b3f-75cf-4c3b-a029-2d61e4e5fa91 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:49.740459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce78b3f-75cf-4c3b-a029-2d61e4e5fa91 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:49.747029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cd5c8-daf4-46ce-9b67-689c54c92e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:49.747029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cd5c8-daf4-46ce-9b67-689c54c92e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:49.747029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cd5c8-daf4-46ce-9b67-689c54c92e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:49.747029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cd5c8-daf4-46ce-9b67-689c54c92e7c admin admin] PATH_INFO: `/v3/projects/d5ed08c77e8247d0abc0ffefda355d6b/users/40fb07886b4c4054a129d34670ef9742/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:49.762757 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cd5c8-daf4-46ce-9b67-689c54c92e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ed08c77e8247d0abc0ffefda355d6b, user_id=40fb07886b4c4054a129d34670ef9742, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:49.765575 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cd5c8-daf4-46ce-9b67-689c54c92e7c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:49.857738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1277/2526] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:49 2022] PUT /identity/v3/projects/d5ed08c77e8247d0abc0ffefda355d6b/users/40fb07886b4c4054a129d34670ef9742/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:49.861861 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00d746-eb9a-4012-8422-89fc1fa74857 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:49.883005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00d746-eb9a-4012-8422-89fc1fa74857 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:50.007057 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b47a10-f9f3-4978-9a38-481eafb7df6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:50.007800 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b47a10-f9f3-4978-9a38-481eafb7df6b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:50.008089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b47a10-f9f3-4978-9a38-481eafb7df6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:50.008366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b47a10-f9f3-4978-9a38-481eafb7df6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:50.009103 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b47a10-f9f3-4978-9a38-481eafb7df6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:50.023828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b47a10-f9f3-4978-9a38-481eafb7df6b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:50.055910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1250/2527] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:50.060111 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cd5c8-daf4-46ce-9b67-689c54c92e7c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:50.074452 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cd5c8-daf4-46ce-9b67-689c54c92e7c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:50.080074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce005a-12ce-4636-ae84-00dd66f48cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:50.080982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce005a-12ce-4636-ae84-00dd66f48cc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:50.084037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce005a-12ce-4636-ae84-00dd66f48cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:50.084037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce005a-12ce-4636-ae84-00dd66f48cc8 admin admin] PATH_INFO: `/v3/projects/d5ed08c77e8247d0abc0ffefda355d6b/users/40fb07886b4c4054a129d34670ef9742/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:50.087233 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce005a-12ce-4636-ae84-00dd66f48cc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ed08c77e8247d0abc0ffefda355d6b, user_id=40fb07886b4c4054a129d34670ef9742, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:50.089248 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce005a-12ce-4636-ae84-00dd66f48cc8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:50.124979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1278/2528] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:50 2022] PUT /identity/v3/projects/d5ed08c77e8247d0abc0ffefda355d6b/users/40fb07886b4c4054a129d34670ef9742/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:21:50.682085 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0247faf-40b5-439c-b7b6-5926a9aae6bc None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:50.682085 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0247faf-40b5-439c-b7b6-5926a9aae6bc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:50.684953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0247faf-40b5-439c-b7b6-5926a9aae6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:50.742118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:50.752034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a0247faf-40b5-439c-b7b6-5926a9aae6bc None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:50.754313 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a0247faf-40b5-439c-b7b6-5926a9aae6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:50.832380 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1251/2529] 10.222.0.67 () {58 vars in 1009 bytes} [Fri Jun 24 15:21:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:21:50.845839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce005a-12ce-4636-ae84-00dd66f48cc8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:50.849936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1dce005a-12ce-4636-ae84-00dd66f48cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:50.990978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce005a-12ce-4636-ae84-00dd66f48cc8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:50.998359 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a88d3-e14e-49f7-9f9c-2185d8fd4be7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:50.998958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a88d3-e14e-49f7-9f9c-2185d8fd4be7 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:51.006897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a88d3-e14e-49f7-9f9c-2185d8fd4be7 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:51.006897 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a88d3-e14e-49f7-9f9c-2185d8fd4be7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:51.016468 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a88d3-e14e-49f7-9f9c-2185d8fd4be7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:51.018238 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a88d3-e14e-49f7-9f9c-2185d8fd4be7 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:51.031013 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1279/2530] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:21:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:21:53.391691 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0247faf-40b5-439c-b7b6-5926a9aae6bc None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:53.393307 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a0247faf-40b5-439c-b7b6-5926a9aae6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:53.542187 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0247faf-40b5-439c-b7b6-5926a9aae6bc None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:53.553146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad065d-c065-409b-b5fb-8b330613614e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:53.554167 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad065d-c065-409b-b5fb-8b330613614e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:53.554487 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad065d-c065-409b-b5fb-8b330613614e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:53.554871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad065d-c065-409b-b5fb-8b330613614e admin admin] PATH_INFO: `/v3/users/33088b427d134fdda32d1b7ed6630388` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:53.558259 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad065d-c065-409b-b5fb-8b330613614e admin admin] RBAC: Authorizing `identity:delete_user(user_id=33088b427d134fdda32d1b7ed6630388)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:53.561329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad065d-c065-409b-b5fb-8b330613614e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:53.821160 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2bad065d-c065-409b-b5fb-8b330613614e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33088b427d134fdda32d1b7ed6630388'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:53.849755 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2bad065d-c065-409b-b5fb-8b330613614e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33088b427d134fdda32d1b7ed6630388'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:53.850938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2bad065d-c065-409b-b5fb-8b330613614e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33088b427d134fdda32d1b7ed6630388'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:53.863369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2bad065d-c065-409b-b5fb-8b330613614e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33088b427d134fdda32d1b7ed6630388'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:53.864891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2bad065d-c065-409b-b5fb-8b330613614e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33088b427d134fdda32d1b7ed6630388'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:53.954819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1252/2531] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:21:53 2022] DELETE /identity/v3/users/33088b427d134fdda32d1b7ed6630388 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:53.957908 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9dbbd-2d4a-46a3-af07-81b6aca4646c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:53.958496 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9dbbd-2d4a-46a3-af07-81b6aca4646c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:53.958929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9dbbd-2d4a-46a3-af07-81b6aca4646c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:54.037074 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5bd9dbbd-2d4a-46a3-af07-81b6aca4646c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:54.049121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5bd9dbbd-2d4a-46a3-af07-81b6aca4646c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:54.219943 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:54.270740 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1280/2532] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:54.289363 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad065d-c065-409b-b5fb-8b330613614e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:54.291526 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2bad065d-c065-409b-b5fb-8b330613614e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:54.322083 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:54.434390 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad065d-c065-409b-b5fb-8b330613614e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:54.446593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e031771-7c79-4a5d-a10e-48fd588ab1b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:54.447304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e031771-7c79-4a5d-a10e-48fd588ab1b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:54.447615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e031771-7c79-4a5d-a10e-48fd588ab1b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:54.451870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e031771-7c79-4a5d-a10e-48fd588ab1b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:54.465795 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e031771-7c79-4a5d-a10e-48fd588ab1b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:54.467615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e031771-7c79-4a5d-a10e-48fd588ab1b7 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:54.479972 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1253/2533] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:21:55.215797 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9dbbd-2d4a-46a3-af07-81b6aca4646c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:55.220001 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5bd9dbbd-2d4a-46a3-af07-81b6aca4646c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:55.234391 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9dbbd-2d4a-46a3-af07-81b6aca4646c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:55.240536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb91532-3196-48b4-aaba-91f28044a983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:55.241633 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb91532-3196-48b4-aaba-91f28044a983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:55.241923 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb91532-3196-48b4-aaba-91f28044a983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:55.242205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb91532-3196-48b4-aaba-91f28044a983 admin admin] PATH_INFO: `/v3/projects/47025fa9a1524b639d2940a0ab34e207` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:55.247512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb91532-3196-48b4-aaba-91f28044a983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47025fa9a1524b639d2940a0ab34e207)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:55.249165 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb91532-3196-48b4-aaba-91f28044a983 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:55.433203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb91532-3196-48b4-aaba-91f28044a983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47025fa9a1524b639d2940a0ab34e207', 'request_id': 'req-adb91532-3196-48b4-aaba-91f28044a983'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:55.441858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb91532-3196-48b4-aaba-91f28044a983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47025fa9a1524b639d2940a0ab34e207', 'request_id': 'req-adb91532-3196-48b4-aaba-91f28044a983'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:55.509258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb91532-3196-48b4-aaba-91f28044a983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47025fa9a1524b639d2940a0ab34e207', 'request_id': 'req-adb91532-3196-48b4-aaba-91f28044a983'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:55.531766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb91532-3196-48b4-aaba-91f28044a983 admin admin] The token cache is being invalidate because project 47025fa9a1524b639d2940a0ab34e207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:21:55.533165 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb91532-3196-48b4-aaba-91f28044a983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:55.534929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb91532-3196-48b4-aaba-91f28044a983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:55.537078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1281/2534] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:21:55 2022] DELETE /identity/v3/projects/47025fa9a1524b639d2940a0ab34e207 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:55.542325 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe44041-68c0-4c0e-bbd8-8d0404587d2c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:55.543149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe44041-68c0-4c0e-bbd8-8d0404587d2c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:55.543149 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe44041-68c0-4c0e-bbd8-8d0404587d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:55.580353 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ebe44041-68c0-4c0e-bbd8-8d0404587d2c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:55.582787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ebe44041-68c0-4c0e-bbd8-8d0404587d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:55.708888 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1254/2535] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:55.716747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8c974-9034-4600-b92d-892dd8dd19ed None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:55.717053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8c974-9034-4600-b92d-892dd8dd19ed None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:55.717433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8c974-9034-4600-b92d-892dd8dd19ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:55.751222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-31a8c974-9034-4600-b92d-892dd8dd19ed None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:55.754583 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-31a8c974-9034-4600-b92d-892dd8dd19ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:55.768184 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe44041-68c0-4c0e-bbd8-8d0404587d2c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:55.773896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1282/2536] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:55.780167 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe44041-68c0-4c0e-bbd8-8d0404587d2c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:55.785271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfb96f-7dec-465e-ae3a-add1065ead4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:55.786736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfb96f-7dec-465e-ae3a-add1065ead4e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:55.789183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfb96f-7dec-465e-ae3a-add1065ead4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:55.792860 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfb96f-7dec-465e-ae3a-add1065ead4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:55.792860 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfb96f-7dec-465e-ae3a-add1065ead4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:21:55.792860 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfb96f-7dec-465e-ae3a-add1065ead4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:55.792860 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfb96f-7dec-465e-ae3a-add1065ead4e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:55.814487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8c974-9034-4600-b92d-892dd8dd19ed None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:55.827169 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1255/2537] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:21:55.833113 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8c974-9034-4600-b92d-892dd8dd19ed None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:55.836333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b14b1d-db1f-4d00-9819-5709aac0fcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:55.837309 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b14b1d-db1f-4d00-9819-5709aac0fcbe admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:55.837692 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b14b1d-db1f-4d00-9819-5709aac0fcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:55.838227 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b14b1d-db1f-4d00-9819-5709aac0fcbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:55.839304 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b14b1d-db1f-4d00-9819-5709aac0fcbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:55.841551 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b14b1d-db1f-4d00-9819-5709aac0fcbe admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:55.943757 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfb96f-7dec-465e-ae3a-add1065ead4e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:55.952737 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1283/2538] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:21:55 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:55.958884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfb96f-7dec-465e-ae3a-add1065ead4e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:56.072862 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8183d-b69c-4e39-acc8-bf46386074d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:56.073625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8183d-b69c-4e39-acc8-bf46386074d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:56.073910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8183d-b69c-4e39-acc8-bf46386074d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:56.074186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8183d-b69c-4e39-acc8-bf46386074d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:56.074966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8183d-b69c-4e39-acc8-bf46386074d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:56.077429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8183d-b69c-4e39-acc8-bf46386074d2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:56.205383 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1256/2539] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:21:55 2022] POST /identity/v3/users => generated 288 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:56.210042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b14b1d-db1f-4d00-9819-5709aac0fcbe admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:56.245893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b14b1d-db1f-4d00-9819-5709aac0fcbe admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:56.260158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40879935-f5e4-4c1b-b1fa-18c8f398c3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:56.261242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40879935-f5e4-4c1b-b1fa-18c8f398c3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:56.269012 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40879935-f5e4-4c1b-b1fa-18c8f398c3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:56.269426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40879935-f5e4-4c1b-b1fa-18c8f398c3d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:56.270092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40879935-f5e4-4c1b-b1fa-18c8f398c3d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:56.271779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40879935-f5e4-4c1b-b1fa-18c8f398c3d7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:56.297457 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1284/2540] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:56.317306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8183d-b69c-4e39-acc8-bf46386074d2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:56.341866 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8183d-b69c-4e39-acc8-bf46386074d2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:56.349889 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795f01f-fd1e-4e24-9055-cd0414778214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:56.350678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795f01f-fd1e-4e24-9055-cd0414778214 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:56.350959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795f01f-fd1e-4e24-9055-cd0414778214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:56.351258 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795f01f-fd1e-4e24-9055-cd0414778214 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:56.352181 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795f01f-fd1e-4e24-9055-cd0414778214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:56.354097 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795f01f-fd1e-4e24-9055-cd0414778214 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:56.384043 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1257/2541] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:56.385277 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40879935-f5e4-4c1b-b1fa-18c8f398c3d7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:56.418039 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40879935-f5e4-4c1b-b1fa-18c8f398c3d7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:56.425932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76168eb-11d9-4cba-ab21-e91baf68c9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:56.433669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76168eb-11d9-4cba-ab21-e91baf68c9c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:56.434127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76168eb-11d9-4cba-ab21-e91baf68c9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:56.434562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76168eb-11d9-4cba-ab21-e91baf68c9c5 admin admin] PATH_INFO: `/v3/projects/7a8881ca032f40ffbc14fc064f5541a8/users/34b6b88bf8314401a3975f1c9d586b94/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:56.466367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76168eb-11d9-4cba-ab21-e91baf68c9c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a8881ca032f40ffbc14fc064f5541a8, user_id=34b6b88bf8314401a3975f1c9d586b94, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:56.471097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76168eb-11d9-4cba-ab21-e91baf68c9c5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:56.592810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1285/2542] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:56 2022] PUT /identity/v3/projects/7a8881ca032f40ffbc14fc064f5541a8/users/34b6b88bf8314401a3975f1c9d586b94/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:56.604055 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460135a-c6c9-43d3-b467-b4efbdb16a24 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:56.605227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460135a-c6c9-43d3-b467-b4efbdb16a24 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:56.605519 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460135a-c6c9-43d3-b467-b4efbdb16a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:56.642619 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9460135a-c6c9-43d3-b467-b4efbdb16a24 None None] MFA Rules not processed for user `34b6b88bf8314401a3975f1c9d586b94`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:56.645015 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9460135a-c6c9-43d3-b467-b4efbdb16a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:56.671448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:56.802771 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1258/2543] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:56.806777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76168eb-11d9-4cba-ab21-e91baf68c9c5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:56.974983 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76168eb-11d9-4cba-ab21-e91baf68c9c5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:56.981732 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019c544-6f8b-46b1-9a50-be97f0f7e17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:56.982832 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019c544-6f8b-46b1-9a50-be97f0f7e17c admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:56.983378 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019c544-6f8b-46b1-9a50-be97f0f7e17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:56.983837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019c544-6f8b-46b1-9a50-be97f0f7e17c admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:56.984697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019c544-6f8b-46b1-9a50-be97f0f7e17c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:56.986846 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019c544-6f8b-46b1-9a50-be97f0f7e17c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:57.011573 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:57.081066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1286/2544] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:21:56 2022] POST /identity/v3/projects => generated 377 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:57.083761 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460135a-c6c9-43d3-b467-b4efbdb16a24 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:57.097044 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460135a-c6c9-43d3-b467-b4efbdb16a24 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:57.206571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1a0bd-71c6-49de-8f6d-a933663fd5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:57.207461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1a0bd-71c6-49de-8f6d-a933663fd5ce admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:57.207915 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1a0bd-71c6-49de-8f6d-a933663fd5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:57.208232 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1a0bd-71c6-49de-8f6d-a933663fd5ce admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:57.209040 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1a0bd-71c6-49de-8f6d-a933663fd5ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:57.212734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1a0bd-71c6-49de-8f6d-a933663fd5ce admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:57.337902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1259/2545] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:21:57 2022] POST /identity/v3/users => generated 289 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:57.338922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019c544-6f8b-46b1-9a50-be97f0f7e17c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:57.358708 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019c544-6f8b-46b1-9a50-be97f0f7e17c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:57.376363 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa942a-7208-4b6b-bedc-9cea881bbc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:57.377094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa942a-7208-4b6b-bedc-9cea881bbc0b admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:57.377421 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa942a-7208-4b6b-bedc-9cea881bbc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:57.379866 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa942a-7208-4b6b-bedc-9cea881bbc0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:57.380641 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa942a-7208-4b6b-bedc-9cea881bbc0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:57.391481 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa942a-7208-4b6b-bedc-9cea881bbc0b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:57.418748 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1287/2546] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:21:57.422702 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1a0bd-71c6-49de-8f6d-a933663fd5ce admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:57.445290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1a0bd-71c6-49de-8f6d-a933663fd5ce admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:57.450415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84b478-396b-4efd-ad1d-a1303e5a17f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:57.451420 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84b478-396b-4efd-ad1d-a1303e5a17f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:57.451808 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84b478-396b-4efd-ad1d-a1303e5a17f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:57.452186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84b478-396b-4efd-ad1d-a1303e5a17f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:57.453031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84b478-396b-4efd-ad1d-a1303e5a17f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:57.454836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84b478-396b-4efd-ad1d-a1303e5a17f6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:57.486958 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa942a-7208-4b6b-bedc-9cea881bbc0b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:57.490596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1260/2547] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:57.501520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa942a-7208-4b6b-bedc-9cea881bbc0b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:57.508152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84f01b-aef9-47aa-b01b-f5119b2cbafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:57.509235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84f01b-aef9-47aa-b01b-f5119b2cbafd admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:57.509535 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84f01b-aef9-47aa-b01b-f5119b2cbafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:57.509930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84f01b-aef9-47aa-b01b-f5119b2cbafd admin admin] PATH_INFO: `/v3/projects/e1318a6a857144a9971c19d99d63cefc/users/c4882043e47145e583a5ef5da49ca3d3/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:57.530355 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b84f01b-aef9-47aa-b01b-f5119b2cbafd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1318a6a857144a9971c19d99d63cefc, user_id=c4882043e47145e583a5ef5da49ca3d3, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:57.533264 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b84f01b-aef9-47aa-b01b-f5119b2cbafd admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:57.656935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1288/2548] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:57 2022] PUT /identity/v3/projects/e1318a6a857144a9971c19d99d63cefc/users/c4882043e47145e583a5ef5da49ca3d3/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:21:57.669567 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0ae74-2cf0-43de-af58-27a82cd6f12f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:57.670124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0ae74-2cf0-43de-af58-27a82cd6f12f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:57.670124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0ae74-2cf0-43de-af58-27a82cd6f12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:57.718944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-dba0ae74-2cf0-43de-af58-27a82cd6f12f None None] MFA Rules not processed for user `c4882043e47145e583a5ef5da49ca3d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:57.720609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dba0ae74-2cf0-43de-af58-27a82cd6f12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:57.870050 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1261/2549] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:57.874625 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84f01b-aef9-47aa-b01b-f5119b2cbafd admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:58.002840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84f01b-aef9-47aa-b01b-f5119b2cbafd admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:58.016450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dbc9c-2f90-4415-8f44-b558c841dae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:58.017592 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dbc9c-2f90-4415-8f44-b558c841dae8 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:58.020520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dbc9c-2f90-4415-8f44-b558c841dae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:58.021125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dbc9c-2f90-4415-8f44-b558c841dae8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:58.022185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dbc9c-2f90-4415-8f44-b558c841dae8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:58.026552 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dbc9c-2f90-4415-8f44-b558c841dae8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:58.764004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1289/2550] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:21:57 2022] POST /identity/v3/projects => generated 377 bytes in 890 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:58.771580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0ae74-2cf0-43de-af58-27a82cd6f12f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:58.796555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0ae74-2cf0-43de-af58-27a82cd6f12f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:58.862819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:58.964936 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9445e-a061-48f5-ac87-d7afa2a0903d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:58.966140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9445e-a061-48f5-ac87-d7afa2a0903d admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:58.966531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9445e-a061-48f5-ac87-d7afa2a0903d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:58.966858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9445e-a061-48f5-ac87-d7afa2a0903d admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:58.967631 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9445e-a061-48f5-ac87-d7afa2a0903d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:58.970386 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9445e-a061-48f5-ac87-d7afa2a0903d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:59.102830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1262/2551] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:21:58 2022] POST /identity/v3/users => generated 295 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:59.106732 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dbc9c-2f90-4415-8f44-b558c841dae8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:59.127154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dbc9c-2f90-4415-8f44-b558c841dae8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:59.161862 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5bac5-27d7-4d0d-a5f5-8c575ac86548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:59.166648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5bac5-27d7-4d0d-a5f5-8c575ac86548 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:59.166648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5bac5-27d7-4d0d-a5f5-8c575ac86548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:59.169685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5bac5-27d7-4d0d-a5f5-8c575ac86548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:59.170628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5bac5-27d7-4d0d-a5f5-8c575ac86548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:59.181423 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5bac5-27d7-4d0d-a5f5-8c575ac86548 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:59.198877 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1290/2552] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:59.208918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9445e-a061-48f5-ac87-d7afa2a0903d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:59.222850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9445e-a061-48f5-ac87-d7afa2a0903d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:59.243408 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99200d3-ba90-4a6b-9411-4ca5dd2c7cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:59.244660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99200d3-ba90-4a6b-9411-4ca5dd2c7cb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:59.245120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99200d3-ba90-4a6b-9411-4ca5dd2c7cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:59.245579 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99200d3-ba90-4a6b-9411-4ca5dd2c7cb3 admin admin] PATH_INFO: `/v3/projects/9349d99e021c46609b4974601328b293/users/65f89b5b7dcd4513b7b194ce5eeea78a/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:59.270355 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99200d3-ba90-4a6b-9411-4ca5dd2c7cb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9349d99e021c46609b4974601328b293, user_id=65f89b5b7dcd4513b7b194ce5eeea78a, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:59.275759 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99200d3-ba90-4a6b-9411-4ca5dd2c7cb3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:59.371146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1263/2553] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:59 2022] PUT /identity/v3/projects/9349d99e021c46609b4974601328b293/users/65f89b5b7dcd4513b7b194ce5eeea78a/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:59.373037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5bac5-27d7-4d0d-a5f5-8c575ac86548 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:59.385478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5bac5-27d7-4d0d-a5f5-8c575ac86548 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:59.548644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85de607-66ec-4527-99bd-78456984db07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:59.549442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85de607-66ec-4527-99bd-78456984db07 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:59.549943 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85de607-66ec-4527-99bd-78456984db07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:59.550362 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85de607-66ec-4527-99bd-78456984db07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:59.551092 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85de607-66ec-4527-99bd-78456984db07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:59.553484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85de607-66ec-4527-99bd-78456984db07 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:59.616654 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1291/2554] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:21:59.621372 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99200d3-ba90-4a6b-9411-4ca5dd2c7cb3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:59.641587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99200d3-ba90-4a6b-9411-4ca5dd2c7cb3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:59.654500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe661b-09cd-443b-bad6-4f676ebec992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:59.655524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe661b-09cd-443b-bad6-4f676ebec992 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:59.655811 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe661b-09cd-443b-bad6-4f676ebec992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:59.659069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe661b-09cd-443b-bad6-4f676ebec992 admin admin] PATH_INFO: `/v3/projects/9349d99e021c46609b4974601328b293/users/65f89b5b7dcd4513b7b194ce5eeea78a/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:59.663620 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe661b-09cd-443b-bad6-4f676ebec992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9349d99e021c46609b4974601328b293, user_id=65f89b5b7dcd4513b7b194ce5eeea78a, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:59.666316 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe661b-09cd-443b-bad6-4f676ebec992 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:59.692075 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1264/2555] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:59 2022] PUT /identity/v3/projects/9349d99e021c46609b4974601328b293/users/65f89b5b7dcd4513b7b194ce5eeea78a/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:59.768207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a02c48-e4b5-4b84-bb86-b68e38308788 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:59.774829 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a02c48-e4b5-4b84-bb86-b68e38308788 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:59.774829 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a02c48-e4b5-4b84-bb86-b68e38308788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:59.837895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-41a02c48-e4b5-4b84-bb86-b68e38308788 None None] MFA Rules not processed for user `65f89b5b7dcd4513b7b194ce5eeea78a`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:59.839197 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-41a02c48-e4b5-4b84-bb86-b68e38308788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:59.897274 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:59.990450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1292/2556] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:21:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:22:00.010047 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe661b-09cd-443b-bad6-4f676ebec992 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:00.012515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5bfe661b-09cd-443b-bad6-4f676ebec992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:00.135303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe661b-09cd-443b-bad6-4f676ebec992 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:00.139458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5651446-13a5-4f72-9ccc-0bcb38eebb59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:00.143038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5651446-13a5-4f72-9ccc-0bcb38eebb59 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:00.143038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5651446-13a5-4f72-9ccc-0bcb38eebb59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:00.143038 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5651446-13a5-4f72-9ccc-0bcb38eebb59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:00.210693 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46246c-b9d1-4b92-ac1d-2cb0961eb061 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:00.210941 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46246c-b9d1-4b92-ac1d-2cb0961eb061 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:00.211239 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46246c-b9d1-4b92-ac1d-2cb0961eb061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:00.291508 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0c46246c-b9d1-4b92-ac1d-2cb0961eb061 None None] MFA Rules not processed for user `40fb07886b4c4054a129d34670ef9742`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:22:00.293469 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0c46246c-b9d1-4b92-ac1d-2cb0961eb061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:00.302886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5651446-13a5-4f72-9ccc-0bcb38eebb59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:00.304528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5651446-13a5-4f72-9ccc-0bcb38eebb59 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:00.342494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1265/2557] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:22:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:22:00.492507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1293/2558] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:22:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:22:00.509348 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5651446-13a5-4f72-9ccc-0bcb38eebb59 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:00.511466 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f5651446-13a5-4f72-9ccc-0bcb38eebb59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:00.613016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:22:00.693212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5651446-13a5-4f72-9ccc-0bcb38eebb59 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:00.710060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56764aa-0ae1-4869-a9bd-9ce06cf465d1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1971c464293f426f97d6cb199e48193b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:00.710687 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56764aa-0ae1-4869-a9bd-9ce06cf465d1 service glance] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:00.710977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56764aa-0ae1-4869-a9bd-9ce06cf465d1 service glance] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:00.711822 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56764aa-0ae1-4869-a9bd-9ce06cf465d1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:00.716805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d56764aa-0ae1-4869-a9bd-9ce06cf465d1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:00.837142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56764aa-0ae1-4869-a9bd-9ce06cf465d1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:00.838712 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56764aa-0ae1-4869-a9bd-9ce06cf465d1 service glance] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:00.865983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1266/2559] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:22:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:22:01.192669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3e883-040f-4f57-9b94-9ab8dc61eba3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:01.196826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3e883-040f-4f57-9b94-9ab8dc61eba3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:01.196826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3e883-040f-4f57-9b94-9ab8dc61eba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:01.223158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-78a3e883-040f-4f57-9b94-9ab8dc61eba3 None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:22:01.224525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-78a3e883-040f-4f57-9b94-9ab8dc61eba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:01.325068 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1294/2560] 10.222.0.67 () {58 vars in 1009 bytes} [Fri Jun 24 15:22:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:22:01.353707 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56764aa-0ae1-4869-a9bd-9ce06cf465d1 service glance] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:01.356010 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d56764aa-0ae1-4869-a9bd-9ce06cf465d1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:01.443633 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56764aa-0ae1-4869-a9bd-9ce06cf465d1 service glance] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:01.449671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88153917-5e4f-4dba-804a-6fad54a48135 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:01.450849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88153917-5e4f-4dba-804a-6fad54a48135 service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:01.451515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88153917-5e4f-4dba-804a-6fad54a48135 service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:01.452036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88153917-5e4f-4dba-804a-6fad54a48135 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:01.462042 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88153917-5e4f-4dba-804a-6fad54a48135 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:01.463944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88153917-5e4f-4dba-804a-6fad54a48135 service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:01.476062 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1267/2561] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:22:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:22:03.870045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3e883-040f-4f57-9b94-9ab8dc61eba3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:03.885849 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3e883-040f-4f57-9b94-9ab8dc61eba3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:03.890740 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584ef5a-b29f-4884-8b31-e1722a8e4926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:03.891730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584ef5a-b29f-4884-8b31-e1722a8e4926 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:03.892194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584ef5a-b29f-4884-8b31-e1722a8e4926 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:03.892473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584ef5a-b29f-4884-8b31-e1722a8e4926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:03.960519 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:22:03.981482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584ef5a-b29f-4884-8b31-e1722a8e4926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:03.983630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584ef5a-b29f-4884-8b31-e1722a8e4926 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:03.999076 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1295/2562] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:22:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:22:06.269575 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88153917-5e4f-4dba-804a-6fad54a48135 service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:06.271073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-88153917-5e4f-4dba-804a-6fad54a48135 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:06.285262 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88153917-5e4f-4dba-804a-6fad54a48135 service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:06.291692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c7ee3-b139-4f9f-b1cb-f8e024614514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:06.292374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c7ee3-b139-4f9f-b1cb-f8e024614514 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:06.292685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c7ee3-b139-4f9f-b1cb-f8e024614514 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:06.293089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c7ee3-b139-4f9f-b1cb-f8e024614514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:06.306444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c7ee3-b139-4f9f-b1cb-f8e024614514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:06.307940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c7ee3-b139-4f9f-b1cb-f8e024614514 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:06.331341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1268/2563] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:22:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:22:11.490521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584ef5a-b29f-4884-8b31-e1722a8e4926 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:11.575025 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584ef5a-b29f-4884-8b31-e1722a8e4926 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:11.580289 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c009d9-dc2f-402c-b71f-8b570d051df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:11.584099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c009d9-dc2f-402c-b71f-8b570d051df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:11.584385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c009d9-dc2f-402c-b71f-8b570d051df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:11.584670 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c009d9-dc2f-402c-b71f-8b570d051df6 admin admin] PATH_INFO: `/v3/users/34b6b88bf8314401a3975f1c9d586b94` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:11.586051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c009d9-dc2f-402c-b71f-8b570d051df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34b6b88bf8314401a3975f1c9d586b94)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:11.588453 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c009d9-dc2f-402c-b71f-8b570d051df6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:11.797393 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-40c009d9-dc2f-402c-b71f-8b570d051df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34b6b88bf8314401a3975f1c9d586b94'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:11.816586 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-40c009d9-dc2f-402c-b71f-8b570d051df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34b6b88bf8314401a3975f1c9d586b94'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:11.893812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-40c009d9-dc2f-402c-b71f-8b570d051df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34b6b88bf8314401a3975f1c9d586b94'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:11.898327 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-40c009d9-dc2f-402c-b71f-8b570d051df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34b6b88bf8314401a3975f1c9d586b94'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:11.909754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-40c009d9-dc2f-402c-b71f-8b570d051df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34b6b88bf8314401a3975f1c9d586b94'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:11.927907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1296/2564] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:22:11 2022] DELETE /identity/v3/users/34b6b88bf8314401a3975f1c9d586b94 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:22:11.933375 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f4661-345f-4843-b60b-9c7b99261bff None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:11.933911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f4661-345f-4843-b60b-9c7b99261bff None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:11.934142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f4661-345f-4843-b60b-9c7b99261bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:11.958732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-877f4661-345f-4843-b60b-9c7b99261bff None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:22:11.961430 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-877f4661-345f-4843-b60b-9c7b99261bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:12.033311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:22:12.089031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1269/2565] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:22:12.095460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c009d9-dc2f-402c-b71f-8b570d051df6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:12.098736 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-40c009d9-dc2f-402c-b71f-8b570d051df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:12.180559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c009d9-dc2f-402c-b71f-8b570d051df6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:12.186931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f01bcb-04d7-4fe3-9993-e72540a5b0b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:12.187697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f01bcb-04d7-4fe3-9993-e72540a5b0b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:12.188047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f01bcb-04d7-4fe3-9993-e72540a5b0b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:12.188975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f01bcb-04d7-4fe3-9993-e72540a5b0b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:12.202274 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f01bcb-04d7-4fe3-9993-e72540a5b0b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:12.203827 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f01bcb-04d7-4fe3-9993-e72540a5b0b9 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:12.214779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1297/2566] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:22:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:22:13.119232 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f4661-345f-4843-b60b-9c7b99261bff None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:13.121166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-877f4661-345f-4843-b60b-9c7b99261bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:13.133971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f4661-345f-4843-b60b-9c7b99261bff None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:13.139596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:13.140840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:13.141319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:13.141625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c admin admin] PATH_INFO: `/v3/projects/7a8881ca032f40ffbc14fc064f5541a8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:13.142838 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a8881ca032f40ffbc14fc064f5541a8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:13.145089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:13.258734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a8881ca032f40ffbc14fc064f5541a8', 'request_id': 'req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:13.269191 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a8881ca032f40ffbc14fc064f5541a8', 'request_id': 'req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:13.345417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a8881ca032f40ffbc14fc064f5541a8', 'request_id': 'req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:13.361035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c admin admin] The token cache is being invalidate because project 7a8881ca032f40ffbc14fc064f5541a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:22:13.361423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:13.363079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:13.367996 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1270/2567] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:22:13 2022] DELETE /identity/v3/projects/7a8881ca032f40ffbc14fc064f5541a8 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:22:13.371380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f01bcb-04d7-4fe3-9993-e72540a5b0b9 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:13.382652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f01bcb-04d7-4fe3-9993-e72540a5b0b9 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:13.386704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:22:13.452852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86b105-f2b2-4c00-baa0-bf41e44080a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:13.453745 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86b105-f2b2-4c00-baa0-bf41e44080a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:13.454046 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86b105-f2b2-4c00-baa0-bf41e44080a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:13.454325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86b105-f2b2-4c00-baa0-bf41e44080a0 admin admin] PATH_INFO: `/v3/users/c4882043e47145e583a5ef5da49ca3d3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:13.455643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86b105-f2b2-4c00-baa0-bf41e44080a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4882043e47145e583a5ef5da49ca3d3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:13.458078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86b105-f2b2-4c00-baa0-bf41e44080a0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:13.644362 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f86b105-f2b2-4c00-baa0-bf41e44080a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4882043e47145e583a5ef5da49ca3d3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:13.659839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f86b105-f2b2-4c00-baa0-bf41e44080a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4882043e47145e583a5ef5da49ca3d3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:13.767442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f86b105-f2b2-4c00-baa0-bf41e44080a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4882043e47145e583a5ef5da49ca3d3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:13.768229 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f86b105-f2b2-4c00-baa0-bf41e44080a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4882043e47145e583a5ef5da49ca3d3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:13.775613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f86b105-f2b2-4c00-baa0-bf41e44080a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4882043e47145e583a5ef5da49ca3d3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:13.781756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1298/2568] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:22:13 2022] DELETE /identity/v3/users/c4882043e47145e583a5ef5da49ca3d3 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:22:14.305161 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:14.305867 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:14.313711 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9422b9b-c610-4fdf-9e33-80734f90cca1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:14.318679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9422b9b-c610-4fdf-9e33-80734f90cca1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:14.318679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9422b9b-c610-4fdf-9e33-80734f90cca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:14.320830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203ab9c-8971-4ea9-acf2-6e5a69d9900c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:14.335318 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a9422b9b-c610-4fdf-9e33-80734f90cca1 None None] MFA Rules not processed for user `e4f136ae627b4a81b6b29c667420f1aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:22:14.336911 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a9422b9b-c610-4fdf-9e33-80734f90cca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:14.389124 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c03d6-7fbc-442b-ada9-b2885133fbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:14.390104 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c03d6-7fbc-442b-ada9-b2885133fbca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:14.390405 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c03d6-7fbc-442b-ada9-b2885133fbca admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:14.390688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c03d6-7fbc-442b-ada9-b2885133fbca admin admin] PATH_INFO: `/v3/projects/e1318a6a857144a9971c19d99d63cefc` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:14.391929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c03d6-7fbc-442b-ada9-b2885133fbca admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1318a6a857144a9971c19d99d63cefc)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:14.395739 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c03d6-7fbc-442b-ada9-b2885133fbca admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:14.398121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1299/2569] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:22:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:22:14.539589 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-824c03d6-7fbc-442b-ada9-b2885133fbca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1318a6a857144a9971c19d99d63cefc', 'request_id': 'req-824c03d6-7fbc-442b-ada9-b2885133fbca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:14.546676 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-824c03d6-7fbc-442b-ada9-b2885133fbca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1318a6a857144a9971c19d99d63cefc', 'request_id': 'req-824c03d6-7fbc-442b-ada9-b2885133fbca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:14.607814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-824c03d6-7fbc-442b-ada9-b2885133fbca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1318a6a857144a9971c19d99d63cefc', 'request_id': 'req-824c03d6-7fbc-442b-ada9-b2885133fbca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:14.620213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:22:14.626436 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-824c03d6-7fbc-442b-ada9-b2885133fbca admin admin] The token cache is being invalidate because project e1318a6a857144a9971c19d99d63cefc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:22:14.626819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-824c03d6-7fbc-442b-ada9-b2885133fbca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:14.628196 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-824c03d6-7fbc-442b-ada9-b2885133fbca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:14.629942 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1271/2570] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:22:14 2022] DELETE /identity/v3/projects/e1318a6a857144a9971c19d99d63cefc => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:22:14.633858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9422b9b-c610-4fdf-9e33-80734f90cca1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:14.636206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a9422b9b-c610-4fdf-9e33-80734f90cca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:14.651472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9422b9b-c610-4fdf-9e33-80734f90cca1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:14.682290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:22:14.710358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0597648-c77f-419f-9ba0-8b41da57eb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:14.711209 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0597648-c77f-419f-9ba0-8b41da57eb12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:14.711512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0597648-c77f-419f-9ba0-8b41da57eb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:14.711940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0597648-c77f-419f-9ba0-8b41da57eb12 admin admin] PATH_INFO: `/v3/users/65f89b5b7dcd4513b7b194ce5eeea78a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:14.713609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0597648-c77f-419f-9ba0-8b41da57eb12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65f89b5b7dcd4513b7b194ce5eeea78a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:14.715568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0597648-c77f-419f-9ba0-8b41da57eb12 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:14.891307 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c0597648-c77f-419f-9ba0-8b41da57eb12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65f89b5b7dcd4513b7b194ce5eeea78a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:14.905213 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c0597648-c77f-419f-9ba0-8b41da57eb12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65f89b5b7dcd4513b7b194ce5eeea78a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:14.973090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c0597648-c77f-419f-9ba0-8b41da57eb12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65f89b5b7dcd4513b7b194ce5eeea78a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:14.973657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c0597648-c77f-419f-9ba0-8b41da57eb12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65f89b5b7dcd4513b7b194ce5eeea78a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:14.981886 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c0597648-c77f-419f-9ba0-8b41da57eb12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65f89b5b7dcd4513b7b194ce5eeea78a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:14.988822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1300/2571] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:22:14 2022] DELETE /identity/v3/users/65f89b5b7dcd4513b7b194ce5eeea78a => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:22:15.466003 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c03d6-7fbc-442b-ada9-b2885133fbca admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:15.469735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-824c03d6-7fbc-442b-ada9-b2885133fbca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:15.482698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c03d6-7fbc-442b-ada9-b2885133fbca admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:15.550427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce69ea3-4988-4719-ae58-0fe0a6697ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:15.550427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce69ea3-4988-4719-ae58-0fe0a6697ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:15.550427 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce69ea3-4988-4719-ae58-0fe0a6697ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:15.550706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce69ea3-4988-4719-ae58-0fe0a6697ca4 admin admin] PATH_INFO: `/v3/projects/9349d99e021c46609b4974601328b293` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:15.551950 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce69ea3-4988-4719-ae58-0fe0a6697ca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9349d99e021c46609b4974601328b293)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:15.553882 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce69ea3-4988-4719-ae58-0fe0a6697ca4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:15.699226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-dce69ea3-4988-4719-ae58-0fe0a6697ca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9349d99e021c46609b4974601328b293', 'request_id': 'req-dce69ea3-4988-4719-ae58-0fe0a6697ca4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:15.706641 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-dce69ea3-4988-4719-ae58-0fe0a6697ca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9349d99e021c46609b4974601328b293', 'request_id': 'req-dce69ea3-4988-4719-ae58-0fe0a6697ca4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:15.784823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-dce69ea3-4988-4719-ae58-0fe0a6697ca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9349d99e021c46609b4974601328b293', 'request_id': 'req-dce69ea3-4988-4719-ae58-0fe0a6697ca4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:15.796384 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-dce69ea3-4988-4719-ae58-0fe0a6697ca4 admin admin] The token cache is being invalidate because project 9349d99e021c46609b4974601328b293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:22:15.796730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-dce69ea3-4988-4719-ae58-0fe0a6697ca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:15.797415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-dce69ea3-4988-4719-ae58-0fe0a6697ca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:22:15.798982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1272/2572] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:22:15 2022] DELETE /identity/v3/projects/9349d99e021c46609b4974601328b293 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:22:20.552041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffc024-b350-4ce2-8c15-71cbfd108564 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:20.552554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffc024-b350-4ce2-8c15-71cbfd108564 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:20.552668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffc024-b350-4ce2-8c15-71cbfd108564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:20.571782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3fffc024-b350-4ce2-8c15-71cbfd108564 None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:22:20.573026 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3fffc024-b350-4ce2-8c15-71cbfd108564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:20.631085 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1301/2573] 10.222.0.67 () {58 vars in 967 bytes} [Fri Jun 24 15:22:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:22:20.644830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce69ea3-4988-4719-ae58-0fe0a6697ca4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:20.645818 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dce69ea3-4988-4719-ae58-0fe0a6697ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:20.716732 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce69ea3-4988-4719-ae58-0fe0a6697ca4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:20.721690 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa415e-fd82-4010-8e28-0ebdc7708c3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:20.722335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa415e-fd82-4010-8e28-0ebdc7708c3a service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:20.722624 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa415e-fd82-4010-8e28-0ebdc7708c3a service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:20.722902 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa415e-fd82-4010-8e28-0ebdc7708c3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:20.730936 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa415e-fd82-4010-8e28-0ebdc7708c3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:20.732556 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa415e-fd82-4010-8e28-0ebdc7708c3a service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:20.741151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1273/2574] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:22:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:22:25.141630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968bb87-e9fb-44b7-a36d-90c8ae18a98c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:25.142113 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968bb87-e9fb-44b7-a36d-90c8ae18a98c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:25.142274 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968bb87-e9fb-44b7-a36d-90c8ae18a98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:25.161315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2968bb87-e9fb-44b7-a36d-90c8ae18a98c None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:22:25.162523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2968bb87-e9fb-44b7-a36d-90c8ae18a98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:25.171730 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1302/2575] 10.222.0.67 () {58 vars in 1009 bytes} [Fri Jun 24 15:22:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:22:25.183660 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa415e-fd82-4010-8e28-0ebdc7708c3a service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:25.194991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa415e-fd82-4010-8e28-0ebdc7708c3a service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:25.199968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43422ec2-e3a7-4931-a504-808b4c7122e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:25.200625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43422ec2-e3a7-4931-a504-808b4c7122e6 service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:25.200904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43422ec2-e3a7-4931-a504-808b4c7122e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:25.201226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43422ec2-e3a7-4931-a504-808b4c7122e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:25.210300 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43422ec2-e3a7-4931-a504-808b4c7122e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:25.211963 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43422ec2-e3a7-4931-a504-808b4c7122e6 service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:25.220138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1274/2576] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:22:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 15:22:35.155726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=67,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:22:35.216817 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:22:38.224703 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968bb87-e9fb-44b7-a36d-90c8ae18a98c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:38.226144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2968bb87-e9fb-44b7-a36d-90c8ae18a98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:38.292163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968bb87-e9fb-44b7-a36d-90c8ae18a98c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:38.296363 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00640ab1-2e7b-45cf-bf03-92eb1116c4a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:38.297101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00640ab1-2e7b-45cf-bf03-92eb1116c4a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:38.297424 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00640ab1-2e7b-45cf-bf03-92eb1116c4a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:38.297753 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00640ab1-2e7b-45cf-bf03-92eb1116c4a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:38.300419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-00640ab1-2e7b-45cf-bf03-92eb1116c4a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:38.370577 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00640ab1-2e7b-45cf-bf03-92eb1116c4a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:38.372081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00640ab1-2e7b-45cf-bf03-92eb1116c4a0 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:38.382572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1303/2577] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:22:38 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:22:48.377922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=50 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:22:51.739443 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877e032-b3ea-4c29-be54-0059e120e0ce None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:51.739984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877e032-b3ea-4c29-be54-0059e120e0ce None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:51.740082 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877e032-b3ea-4c29-be54-0059e120e0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:51.757545 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b877e032-b3ea-4c29-be54-0059e120e0ce None None] MFA Rules not processed for user `2321d0531bba403eb552c3a1dbb7484c`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:22:51.758727 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b877e032-b3ea-4c29-be54-0059e120e0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:51.814260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1275/2578] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:22:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:23:01.807151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=25 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:23:04.899760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00640ab1-2e7b-45cf-bf03-92eb1116c4a0 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:23:04.901195 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-00640ab1-2e7b-45cf-bf03-92eb1116c4a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:23:04.913938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00640ab1-2e7b-45cf-bf03-92eb1116c4a0 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:23:04.917692 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bb72e-d8c5-4051-830d-855f053ebf5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:23:04.920967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bb72e-d8c5-4051-830d-855f053ebf5b service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:23:04.920967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bb72e-d8c5-4051-830d-855f053ebf5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:23:04.920967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bb72e-d8c5-4051-830d-855f053ebf5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:23:04.920967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3f2bb72e-d8c5-4051-830d-855f053ebf5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:23:04.985707 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bb72e-d8c5-4051-830d-855f053ebf5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:23:04.987198 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bb72e-d8c5-4051-830d-855f053ebf5b service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:23:04.996725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1304/2579] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:23:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:23:14.992317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=30 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:23:15.190571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f199c8-7421-4707-84c5-85fb0e12f0dd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:23:15.191232 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f199c8-7421-4707-84c5-85fb0e12f0dd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:23:15.191232 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f199c8-7421-4707-84c5-85fb0e12f0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:23:15.208645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-71f199c8-7421-4707-84c5-85fb0e12f0dd None None] MFA Rules not processed for user `b67761f9b13340a090efc1cb07714912`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:23:15.209864 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-71f199c8-7421-4707-84c5-85fb0e12f0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:23:15.264027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1276/2580] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:23:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:23:25.257491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=25 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:23:32.297537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafd273-d058-4e64-a0cf-71be1ee7302c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:23:32.297537 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafd273-d058-4e64-a0cf-71be1ee7302c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:23:32.298175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafd273-d058-4e64-a0cf-71be1ee7302c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:23:32.315142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1fafd273-d058-4e64-a0cf-71be1ee7302c None None] MFA Rules not processed for user `206ca9f278fc47e4a9a4f720083e91f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:23:32.316394 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1fafd273-d058-4e64-a0cf-71be1ee7302c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:23:32.374309 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1305/2581] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:23:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:23:32.412010 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f199c8-7421-4707-84c5-85fb0e12f0dd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:23:32.413483 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-71f199c8-7421-4707-84c5-85fb0e12f0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:23:32.469376 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f199c8-7421-4707-84c5-85fb0e12f0dd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:23:32.472912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec326b-5ea8-48b0-80a5-c869ddfa3f6f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1971c464293f426f97d6cb199e48193b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:23:32.473661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec326b-5ea8-48b0-80a5-c869ddfa3f6f service glance] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:23:32.473939 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec326b-5ea8-48b0-80a5-c869ddfa3f6f service glance] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:23:32.474216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec326b-5ea8-48b0-80a5-c869ddfa3f6f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:23:32.476022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-37ec326b-5ea8-48b0-80a5-c869ddfa3f6f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:23:32.549283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec326b-5ea8-48b0-80a5-c869ddfa3f6f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:23:32.550712 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec326b-5ea8-48b0-80a5-c869ddfa3f6f service glance] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:23:32.562740 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1277/2582] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:23:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:23:35.467070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fafd273-d058-4e64-a0cf-71be1ee7302c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:23:35.477639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fafd273-d058-4e64-a0cf-71be1ee7302c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:23:35.481242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd416910-53a1-488a-be23-f0693ba0b2ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:23:35.481880 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd416910-53a1-488a-be23-f0693ba0b2ad service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:23:35.482165 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd416910-53a1-488a-be23-f0693ba0b2ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:23:35.482442 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd416910-53a1-488a-be23-f0693ba0b2ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:23:35.484431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dd416910-53a1-488a-be23-f0693ba0b2ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:23:35.498532 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd416910-53a1-488a-be23-f0693ba0b2ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:23:35.500001 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd416910-53a1-488a-be23-f0693ba0b2ad service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:23:35.511095 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1306/2583] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:23:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:23:42.558052 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=54 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:23:43.708500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec326b-5ea8-48b0-80a5-c869ddfa3f6f service glance] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:23:43.710923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-37ec326b-5ea8-48b0-80a5-c869ddfa3f6f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:23:43.725159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec326b-5ea8-48b0-80a5-c869ddfa3f6f service glance] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:23:43.730650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820441c-0bf9-4c4b-afcb-5700d3086c3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:23:43.731413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820441c-0bf9-4c4b-afcb-5700d3086c3a service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:23:43.731702 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820441c-0bf9-4c4b-afcb-5700d3086c3a service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:23:43.731984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820441c-0bf9-4c4b-afcb-5700d3086c3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:23:43.733871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4820441c-0bf9-4c4b-afcb-5700d3086c3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:23:43.744579 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820441c-0bf9-4c4b-afcb-5700d3086c3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:23:43.746141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820441c-0bf9-4c4b-afcb-5700d3086c3a service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:23:43.755252 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1278/2584] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:23:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:23:45.506644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=31 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:23:53.361485 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd416910-53a1-488a-be23-f0693ba0b2ad service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:23:53.370404 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd416910-53a1-488a-be23-f0693ba0b2ad service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:23:53.374362 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e7078-3cf3-4718-bc9a-87a47d620f13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:23:53.381260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e7078-3cf3-4718-bc9a-87a47d620f13 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:23:53.381260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e7078-3cf3-4718-bc9a-87a47d620f13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:23:53.381260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e7078-3cf3-4718-bc9a-87a47d620f13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:23:53.381260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e34e7078-3cf3-4718-bc9a-87a47d620f13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:23:53.391262 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e7078-3cf3-4718-bc9a-87a47d620f13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:23:53.392758 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e7078-3cf3-4718-bc9a-87a47d620f13 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:23:53.402365 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1307/2585] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:23:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:23:53.751009 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=6 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:23:59.024016 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820441c-0bf9-4c4b-afcb-5700d3086c3a service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:23:59.025645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4820441c-0bf9-4c4b-afcb-5700d3086c3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:23:59.091608 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820441c-0bf9-4c4b-afcb-5700d3086c3a service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:23:59.096347 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e881c1-7b93-423a-b426-d79013d76616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:23:59.097228 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e881c1-7b93-423a-b426-d79013d76616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:23:59.097515 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e881c1-7b93-423a-b426-d79013d76616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:23:59.097797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e881c1-7b93-423a-b426-d79013d76616 admin admin] PATH_INFO: `/v3/users/2321d0531bba403eb552c3a1dbb7484c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:23:59.098988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e881c1-7b93-423a-b426-d79013d76616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2321d0531bba403eb552c3a1dbb7484c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:23:59.100961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e881c1-7b93-423a-b426-d79013d76616 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:23:59.304338 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b5e881c1-7b93-423a-b426-d79013d76616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2321d0531bba403eb552c3a1dbb7484c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:23:59.316984 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b5e881c1-7b93-423a-b426-d79013d76616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2321d0531bba403eb552c3a1dbb7484c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:23:59.317668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b5e881c1-7b93-423a-b426-d79013d76616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2321d0531bba403eb552c3a1dbb7484c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:23:59.324479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b5e881c1-7b93-423a-b426-d79013d76616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2321d0531bba403eb552c3a1dbb7484c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:23:59.325108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b5e881c1-7b93-423a-b426-d79013d76616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2321d0531bba403eb552c3a1dbb7484c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:23:59.386861 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1279/2586] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:23:59 2022] DELETE /identity/v3/users/2321d0531bba403eb552c3a1dbb7484c => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:23:59.664666 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e7078-3cf3-4718-bc9a-87a47d620f13 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:23:59.677151 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e7078-3cf3-4718-bc9a-87a47d620f13 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:23:59.734831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f2676-6013-488f-ac59-a6042e651d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:23:59.735729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f2676-6013-488f-ac59-a6042e651d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:23:59.736017 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f2676-6013-488f-ac59-a6042e651d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:23:59.736298 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f2676-6013-488f-ac59-a6042e651d8c admin admin] PATH_INFO: `/v3/projects/90288e11d18a4c58b77d848fdd630535` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:23:59.737482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f2676-6013-488f-ac59-a6042e651d8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=90288e11d18a4c58b77d848fdd630535)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:23:59.739418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f2676-6013-488f-ac59-a6042e651d8c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:23:59.823480 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ae1f2676-6013-488f-ac59-a6042e651d8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90288e11d18a4c58b77d848fdd630535', 'request_id': 'req-ae1f2676-6013-488f-ac59-a6042e651d8c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:23:59.829607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ae1f2676-6013-488f-ac59-a6042e651d8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90288e11d18a4c58b77d848fdd630535', 'request_id': 'req-ae1f2676-6013-488f-ac59-a6042e651d8c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:23:59.883815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ae1f2676-6013-488f-ac59-a6042e651d8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90288e11d18a4c58b77d848fdd630535', 'request_id': 'req-ae1f2676-6013-488f-ac59-a6042e651d8c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:23:59.894487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ae1f2676-6013-488f-ac59-a6042e651d8c admin admin] The token cache is being invalidate because project 90288e11d18a4c58b77d848fdd630535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:23:59.894779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ae1f2676-6013-488f-ac59-a6042e651d8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:23:59.895438 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ae1f2676-6013-488f-ac59-a6042e651d8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:23:59.896968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1308/2587] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:23:59 2022] DELETE /identity/v3/projects/90288e11d18a4c58b77d848fdd630535 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:23:59.903242 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce59f9f-593c-4e28-9adb-c85e16bd0e1d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:23:59.903526 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce59f9f-593c-4e28-9adb-c85e16bd0e1d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:23:59.903799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce59f9f-593c-4e28-9adb-c85e16bd0e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:23:59.921348 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5ce59f9f-593c-4e28-9adb-c85e16bd0e1d None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:23:59.922547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5ce59f9f-593c-4e28-9adb-c85e16bd0e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:23:59.980594 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1280/2588] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:23:59.988521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b464ba-85fa-4bbc-a469-f9f5497e3afd None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:23:59.988815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b464ba-85fa-4bbc-a469-f9f5497e3afd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:23:59.989099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b464ba-85fa-4bbc-a469-f9f5497e3afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:00.006986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-49b464ba-85fa-4bbc-a469-f9f5497e3afd None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:00.008229 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-49b464ba-85fa-4bbc-a469-f9f5497e3afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:00.016895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1309/2589] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:00.020886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce59f9f-593c-4e28-9adb-c85e16bd0e1d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:00.031113 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce59f9f-593c-4e28-9adb-c85e16bd0e1d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:00.035619 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88a19f-e046-4383-90d0-d6e74b760747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:00.036266 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88a19f-e046-4383-90d0-d6e74b760747 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:00.036547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88a19f-e046-4383-90d0-d6e74b760747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:00.036820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88a19f-e046-4383-90d0-d6e74b760747 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:00.037563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88a19f-e046-4383-90d0-d6e74b760747 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:24:00.037868 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88a19f-e046-4383-90d0-d6e74b760747 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:00.039486 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88a19f-e046-4383-90d0-d6e74b760747 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:00.050719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1281/2590] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:24:00.054819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b464ba-85fa-4bbc-a469-f9f5497e3afd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:00.063872 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b464ba-85fa-4bbc-a469-f9f5497e3afd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:00.068443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f3b44-bb78-4ebd-bade-dfaaea061810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:00.069085 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f3b44-bb78-4ebd-bade-dfaaea061810 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:00.069363 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f3b44-bb78-4ebd-bade-dfaaea061810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:00.069637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f3b44-bb78-4ebd-bade-dfaaea061810 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:00.070320 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f3b44-bb78-4ebd-bade-dfaaea061810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:00.072868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f3b44-bb78-4ebd-bade-dfaaea061810 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:00.145331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1310/2591] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:00 2022] POST /identity/v3/projects => generated 379 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:00.149867 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88a19f-e046-4383-90d0-d6e74b760747 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:00.160329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88a19f-e046-4383-90d0-d6e74b760747 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:00.164306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:00.216294 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1495952-2f60-44b8-8dd6-b139e645d3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:00.216955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1495952-2f60-44b8-8dd6-b139e645d3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:00.217234 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1495952-2f60-44b8-8dd6-b139e645d3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:00.217506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1495952-2f60-44b8-8dd6-b139e645d3c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:00.218183 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1495952-2f60-44b8-8dd6-b139e645d3c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:00.220227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1495952-2f60-44b8-8dd6-b139e645d3c1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:00.329946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1282/2592] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:00 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:00.334071 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f3b44-bb78-4ebd-bade-dfaaea061810 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:00.344403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f3b44-bb78-4ebd-bade-dfaaea061810 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:00.349078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba763f-c475-41eb-987a-b990dd758408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:00.349783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba763f-c475-41eb-987a-b990dd758408 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:00.350019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba763f-c475-41eb-987a-b990dd758408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:00.350311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba763f-c475-41eb-987a-b990dd758408 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:00.351017 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba763f-c475-41eb-987a-b990dd758408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:00.353343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba763f-c475-41eb-987a-b990dd758408 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:00.363590 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1311/2593] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:00.367524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1495952-2f60-44b8-8dd6-b139e645d3c1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:00.378136 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1495952-2f60-44b8-8dd6-b139e645d3c1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:00.382780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f6d0a-f222-4348-8d6a-091150d2b743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:00.384095 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f6d0a-f222-4348-8d6a-091150d2b743 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:00.384417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f6d0a-f222-4348-8d6a-091150d2b743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:00.384774 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f6d0a-f222-4348-8d6a-091150d2b743 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:00.385481 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f6d0a-f222-4348-8d6a-091150d2b743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:00.387056 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f6d0a-f222-4348-8d6a-091150d2b743 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:00.396966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1283/2594] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:00.401450 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba763f-c475-41eb-987a-b990dd758408 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:00.411593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba763f-c475-41eb-987a-b990dd758408 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:00.416306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97617db3-d9f5-412f-804a-23aeb8938b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:00.418035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97617db3-d9f5-412f-804a-23aeb8938b35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:00.418035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97617db3-d9f5-412f-804a-23aeb8938b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:00.418035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97617db3-d9f5-412f-804a-23aeb8938b35 admin admin] PATH_INFO: `/v3/projects/06aac4c0b410452fa71221434149e3b7/users/6f9c73ca3c364117993dac93acf5a737/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:00.432599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97617db3-d9f5-412f-804a-23aeb8938b35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06aac4c0b410452fa71221434149e3b7, user_id=6f9c73ca3c364117993dac93acf5a737, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:00.435265 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97617db3-d9f5-412f-804a-23aeb8938b35 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:00.527015 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1312/2595] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:00 2022] PUT /identity/v3/projects/06aac4c0b410452fa71221434149e3b7/users/6f9c73ca3c364117993dac93acf5a737/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:00.532044 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237857a-79f9-4902-83ac-59ba975c3af4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:00.532377 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237857a-79f9-4902-83ac-59ba975c3af4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:00.532736 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237857a-79f9-4902-83ac-59ba975c3af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:00.550787 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a237857a-79f9-4902-83ac-59ba975c3af4 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:00.552025 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a237857a-79f9-4902-83ac-59ba975c3af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:00.612898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1284/2596] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:00.640406 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97617db3-d9f5-412f-804a-23aeb8938b35 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:00.642240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-97617db3-d9f5-412f-804a-23aeb8938b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:00.656971 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:00.707172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97617db3-d9f5-412f-804a-23aeb8938b35 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:00.711894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28e89e-3b26-450a-a5cc-cfd48f505120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:00.712721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28e89e-3b26-450a-a5cc-cfd48f505120 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:00.714922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28e89e-3b26-450a-a5cc-cfd48f505120 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:00.714922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28e89e-3b26-450a-a5cc-cfd48f505120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:00.723333 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28e89e-3b26-450a-a5cc-cfd48f505120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:00.724888 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28e89e-3b26-450a-a5cc-cfd48f505120 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:00.736316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1313/2597] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:06.148898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd209b-b5ff-4b0f-aab4-063ac39b248f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:06.149542 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd209b-b5ff-4b0f-aab4-063ac39b248f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:06.149542 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd209b-b5ff-4b0f-aab4-063ac39b248f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:06.178223 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-20dd209b-b5ff-4b0f-aab4-063ac39b248f None None] MFA Rules not processed for user `6f9c73ca3c364117993dac93acf5a737`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:06.179462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-20dd209b-b5ff-4b0f-aab4-063ac39b248f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:06.249679 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1285/2598] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:24:06.250818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28e89e-3b26-450a-a5cc-cfd48f505120 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:06.265812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28e89e-3b26-450a-a5cc-cfd48f505120 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:06.276385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8c162-941e-4e95-8196-6458c6796483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:06.277075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8c162-941e-4e95-8196-6458c6796483 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:06.277356 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8c162-941e-4e95-8196-6458c6796483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:06.277626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8c162-941e-4e95-8196-6458c6796483 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:06.278337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8c162-941e-4e95-8196-6458c6796483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:06.280486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8c162-941e-4e95-8196-6458c6796483 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:06.397743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1314/2599] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:06 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:06.399143 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd209b-b5ff-4b0f-aab4-063ac39b248f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:06.411921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd209b-b5ff-4b0f-aab4-063ac39b248f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:06.429151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:06.476705 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf67e6-ea81-45f7-8945-2202a00f9076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:06.479995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf67e6-ea81-45f7-8945-2202a00f9076 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:06.479995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf67e6-ea81-45f7-8945-2202a00f9076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:06.479995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf67e6-ea81-45f7-8945-2202a00f9076 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:06.479995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf67e6-ea81-45f7-8945-2202a00f9076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:06.480625 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf67e6-ea81-45f7-8945-2202a00f9076 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:06.565663 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1286/2600] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:06 2022] POST /identity/v3/users => generated 296 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:06.566968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8c162-941e-4e95-8196-6458c6796483 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:06.578242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8c162-941e-4e95-8196-6458c6796483 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:06.582971 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cb62b-d4a1-4bb5-a29f-3fb836046cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:06.584035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cb62b-d4a1-4bb5-a29f-3fb836046cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:06.584035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cb62b-d4a1-4bb5-a29f-3fb836046cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:06.584269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cb62b-d4a1-4bb5-a29f-3fb836046cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:06.585030 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cb62b-d4a1-4bb5-a29f-3fb836046cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:06.586631 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cb62b-d4a1-4bb5-a29f-3fb836046cc2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:06.598204 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1315/2601] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:06.602758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf67e6-ea81-45f7-8945-2202a00f9076 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:06.612940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf67e6-ea81-45f7-8945-2202a00f9076 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:06.618377 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df49889-076e-4ff4-a33d-347a82e6c9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:06.619425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df49889-076e-4ff4-a33d-347a82e6c9ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:06.619709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df49889-076e-4ff4-a33d-347a82e6c9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:06.619985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df49889-076e-4ff4-a33d-347a82e6c9ba admin admin] PATH_INFO: `/v3/projects/d3d6a581ccf64fa6a40a8fbad19a6567/users/1bd375ff8eea4375836e31dee2e9b364/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:06.634605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df49889-076e-4ff4-a33d-347a82e6c9ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d6a581ccf64fa6a40a8fbad19a6567, user_id=1bd375ff8eea4375836e31dee2e9b364, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:06.637302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df49889-076e-4ff4-a33d-347a82e6c9ba admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:06.707537 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1287/2602] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:06 2022] PUT /identity/v3/projects/d3d6a581ccf64fa6a40a8fbad19a6567/users/1bd375ff8eea4375836e31dee2e9b364/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:06.711800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cb62b-d4a1-4bb5-a29f-3fb836046cc2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:06.722538 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cb62b-d4a1-4bb5-a29f-3fb836046cc2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:06.780760 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bc329-d494-44f4-b0ad-6e5b97a9f3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:06.781436 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bc329-d494-44f4-b0ad-6e5b97a9f3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:06.781721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bc329-d494-44f4-b0ad-6e5b97a9f3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:06.781998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bc329-d494-44f4-b0ad-6e5b97a9f3a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:06.782725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2bc329-d494-44f4-b0ad-6e5b97a9f3a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:06.784349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2bc329-d494-44f4-b0ad-6e5b97a9f3a8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:06.794965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1316/2603] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:06.799242 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df49889-076e-4ff4-a33d-347a82e6c9ba admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:06.810527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df49889-076e-4ff4-a33d-347a82e6c9ba admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:06.815426 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3f4c3-3549-41f2-965c-7c461de49fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:06.816453 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3f4c3-3549-41f2-965c-7c461de49fd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:06.816842 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3f4c3-3549-41f2-965c-7c461de49fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:06.817128 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3f4c3-3549-41f2-965c-7c461de49fd7 admin admin] PATH_INFO: `/v3/projects/d3d6a581ccf64fa6a40a8fbad19a6567/users/1bd375ff8eea4375836e31dee2e9b364/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:06.819271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3f4c3-3549-41f2-965c-7c461de49fd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d6a581ccf64fa6a40a8fbad19a6567, user_id=1bd375ff8eea4375836e31dee2e9b364, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:06.822287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3f4c3-3549-41f2-965c-7c461de49fd7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:06.833138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1288/2604] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:06 2022] PUT /identity/v3/projects/d3d6a581ccf64fa6a40a8fbad19a6567/users/1bd375ff8eea4375836e31dee2e9b364/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:06.902087 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bc329-d494-44f4-b0ad-6e5b97a9f3a8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:06.903913 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-aa2bc329-d494-44f4-b0ad-6e5b97a9f3a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:06.971211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bc329-d494-44f4-b0ad-6e5b97a9f3a8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:07.006366 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:07.026069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9122e12-bd5b-4f06-b294-c6473cb577c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:07.029099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9122e12-bd5b-4f06-b294-c6473cb577c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:07.029099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9122e12-bd5b-4f06-b294-c6473cb577c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:07.029099 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9122e12-bd5b-4f06-b294-c6473cb577c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:07.029613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b9122e12-bd5b-4f06-b294-c6473cb577c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:07.091441 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9122e12-bd5b-4f06-b294-c6473cb577c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:07.092831 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9122e12-bd5b-4f06-b294-c6473cb577c7 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:07.103725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1317/2605] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:12.799290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324adf5f-4f8d-48f1-8277-07513fd6685d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:12.799870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324adf5f-4f8d-48f1-8277-07513fd6685d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:12.799870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324adf5f-4f8d-48f1-8277-07513fd6685d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:12.829815 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-324adf5f-4f8d-48f1-8277-07513fd6685d None None] MFA Rules not processed for user `1bd375ff8eea4375836e31dee2e9b364`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:12.831221 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-324adf5f-4f8d-48f1-8277-07513fd6685d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:12.906094 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1289/2606] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:12.921632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9122e12-bd5b-4f06-b294-c6473cb577c7 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:12.923128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b9122e12-bd5b-4f06-b294-c6473cb577c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:13.005334 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9122e12-bd5b-4f06-b294-c6473cb577c7 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:13.011950 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c214a-432e-46cb-bb0a-3818698f63b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:13.012719 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c214a-432e-46cb-bb0a-3818698f63b8 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:13.012964 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c214a-432e-46cb-bb0a-3818698f63b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:13.013250 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c214a-432e-46cb-bb0a-3818698f63b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:13.077088 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c214a-432e-46cb-bb0a-3818698f63b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:13.078762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c214a-432e-46cb-bb0a-3818698f63b8 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:13.090622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1318/2607] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:14.749761 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324adf5f-4f8d-48f1-8277-07513fd6685d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:14.778337 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324adf5f-4f8d-48f1-8277-07513fd6685d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:14.789781 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4ab4f-45d5-4d1c-969f-866263cb87dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:14.790661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4ab4f-45d5-4d1c-969f-866263cb87dc service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:14.791451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4ab4f-45d5-4d1c-969f-866263cb87dc service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:14.791874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4ab4f-45d5-4d1c-969f-866263cb87dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:14.798605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9fe4ab4f-45d5-4d1c-969f-866263cb87dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:14.814405 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4ab4f-45d5-4d1c-969f-866263cb87dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:14.816310 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4ab4f-45d5-4d1c-969f-866263cb87dc service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:14.829921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1290/2608] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:23.084043 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=88 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:24.824751 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:29.964322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c214a-432e-46cb-bb0a-3818698f63b8 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:29.965755 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b42c214a-432e-46cb-bb0a-3818698f63b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:30.020423 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c214a-432e-46cb-bb0a-3818698f63b8 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:30.024756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bed8c-577d-41f2-831e-21b3e64edfe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:30.025804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bed8c-577d-41f2-831e-21b3e64edfe3 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:30.026084 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bed8c-577d-41f2-831e-21b3e64edfe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:30.026358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bed8c-577d-41f2-831e-21b3e64edfe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:30.028132 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-049bed8c-577d-41f2-831e-21b3e64edfe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:30.089611 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049bed8c-577d-41f2-831e-21b3e64edfe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:30.091075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049bed8c-577d-41f2-831e-21b3e64edfe3 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:30.102093 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1319/2609] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:32.820981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af400b8-14be-4a30-b652-7e0aaba1b78e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:32.821838 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af400b8-14be-4a30-b652-7e0aaba1b78e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:32.822260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af400b8-14be-4a30-b652-7e0aaba1b78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:32.847738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1af400b8-14be-4a30-b652-7e0aaba1b78e None None] MFA Rules not processed for user `6f9c73ca3c364117993dac93acf5a737`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:32.848965 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1af400b8-14be-4a30-b652-7e0aaba1b78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:32.857014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1291/2610] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:33.238896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bed8c-577d-41f2-831e-21b3e64edfe3 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:33.248557 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bed8c-577d-41f2-831e-21b3e64edfe3 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:33.253180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab569f19-cb73-4fc1-966c-128ebaf20cdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:33.254029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab569f19-cb73-4fc1-966c-128ebaf20cdf service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:33.254507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab569f19-cb73-4fc1-966c-128ebaf20cdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:33.255311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab569f19-cb73-4fc1-966c-128ebaf20cdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:33.257917 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ab569f19-cb73-4fc1-966c-128ebaf20cdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:33.269351 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab569f19-cb73-4fc1-966c-128ebaf20cdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:33.270796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab569f19-cb73-4fc1-966c-128ebaf20cdf service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:33.281679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1320/2611] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:33.629828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af400b8-14be-4a30-b652-7e0aaba1b78e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:33.631376 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1af400b8-14be-4a30-b652-7e0aaba1b78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:33.705931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af400b8-14be-4a30-b652-7e0aaba1b78e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:33.711268 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01834259-0e7e-4164-983d-2cce198fb16e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:33.711974 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01834259-0e7e-4164-983d-2cce198fb16e service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:33.712260 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01834259-0e7e-4164-983d-2cce198fb16e service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:33.712558 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01834259-0e7e-4164-983d-2cce198fb16e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:33.714354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-01834259-0e7e-4164-983d-2cce198fb16e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:33.725698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01834259-0e7e-4164-983d-2cce198fb16e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:33.728536 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01834259-0e7e-4164-983d-2cce198fb16e service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:33.738790 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab569f19-cb73-4fc1-966c-128ebaf20cdf service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:33.740432 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ab569f19-cb73-4fc1-966c-128ebaf20cdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:33.740553 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1292/2612] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:33.760628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab569f19-cb73-4fc1-966c-128ebaf20cdf service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:33.765278 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc31c95-5fae-4ca3-adc9-44b165abec23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:33.766186 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc31c95-5fae-4ca3-adc9-44b165abec23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:33.766478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc31c95-5fae-4ca3-adc9-44b165abec23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:33.771263 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc31c95-5fae-4ca3-adc9-44b165abec23 admin admin] PATH_INFO: `/v3/users/b67761f9b13340a090efc1cb07714912` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:33.771263 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc31c95-5fae-4ca3-adc9-44b165abec23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b67761f9b13340a090efc1cb07714912)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:33.771263 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc31c95-5fae-4ca3-adc9-44b165abec23 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:33.971647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6fc31c95-5fae-4ca3-adc9-44b165abec23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b67761f9b13340a090efc1cb07714912'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:33.984535 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6fc31c95-5fae-4ca3-adc9-44b165abec23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b67761f9b13340a090efc1cb07714912'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:34.062800 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01834259-0e7e-4164-983d-2cce198fb16e service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:34.072152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6fc31c95-5fae-4ca3-adc9-44b165abec23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b67761f9b13340a090efc1cb07714912'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:34.072781 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6fc31c95-5fae-4ca3-adc9-44b165abec23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b67761f9b13340a090efc1cb07714912'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:34.074981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01834259-0e7e-4164-983d-2cce198fb16e service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:34.080530 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0d313-3a2e-48d7-82b7-e70f117e1d1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:34.080852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6fc31c95-5fae-4ca3-adc9-44b165abec23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b67761f9b13340a090efc1cb07714912'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:34.081379 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0d313-3a2e-48d7-82b7-e70f117e1d1a service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:34.081918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0d313-3a2e-48d7-82b7-e70f117e1d1a service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:34.082182 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0d313-3a2e-48d7-82b7-e70f117e1d1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:34.084467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-26e0d313-3a2e-48d7-82b7-e70f117e1d1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:34.095618 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1321/2613] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:33 2022] DELETE /identity/v3/users/b67761f9b13340a090efc1cb07714912 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:24:34.114457 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0d313-3a2e-48d7-82b7-e70f117e1d1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:34.116212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0d313-3a2e-48d7-82b7-e70f117e1d1a service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:34.127440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1293/2614] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:24:34.385781 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc31c95-5fae-4ca3-adc9-44b165abec23 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:34.390746 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6fc31c95-5fae-4ca3-adc9-44b165abec23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:34.400932 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:34.456603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc31c95-5fae-4ca3-adc9-44b165abec23 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:34.461691 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4043799-64d9-47c0-a020-47e6b4189bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:34.462600 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4043799-64d9-47c0-a020-47e6b4189bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:34.462886 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4043799-64d9-47c0-a020-47e6b4189bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:34.463162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4043799-64d9-47c0-a020-47e6b4189bf2 admin admin] PATH_INFO: `/v3/projects/b6bf0e1123fa47a7844e09127fb16d6e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:34.464370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4043799-64d9-47c0-a020-47e6b4189bf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6bf0e1123fa47a7844e09127fb16d6e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:34.466306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4043799-64d9-47c0-a020-47e6b4189bf2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:34.595391 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4043799-64d9-47c0-a020-47e6b4189bf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6bf0e1123fa47a7844e09127fb16d6e', 'request_id': 'req-c4043799-64d9-47c0-a020-47e6b4189bf2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:34.602061 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4043799-64d9-47c0-a020-47e6b4189bf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6bf0e1123fa47a7844e09127fb16d6e', 'request_id': 'req-c4043799-64d9-47c0-a020-47e6b4189bf2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:34.680637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4043799-64d9-47c0-a020-47e6b4189bf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6bf0e1123fa47a7844e09127fb16d6e', 'request_id': 'req-c4043799-64d9-47c0-a020-47e6b4189bf2'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:34.691595 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4043799-64d9-47c0-a020-47e6b4189bf2 admin admin] The token cache is being invalidate because project b6bf0e1123fa47a7844e09127fb16d6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:34.691884 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4043799-64d9-47c0-a020-47e6b4189bf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:34.692501 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4043799-64d9-47c0-a020-47e6b4189bf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:34.694027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1322/2615] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:34 2022] DELETE /identity/v3/projects/b6bf0e1123fa47a7844e09127fb16d6e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:34.699704 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a1e9f-b499-4e05-955c-abef3ac63576 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:34.699994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a1e9f-b499-4e05-955c-abef3ac63576 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:34.700272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a1e9f-b499-4e05-955c-abef3ac63576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:34.718497 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-382a1e9f-b499-4e05-955c-abef3ac63576 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:34.719715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-382a1e9f-b499-4e05-955c-abef3ac63576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:34.778989 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1294/2616] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:34.786545 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4bc16-60c9-49f1-a412-86f5b93694e9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:34.786863 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4bc16-60c9-49f1-a412-86f5b93694e9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:34.787142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4bc16-60c9-49f1-a412-86f5b93694e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:34.806311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6cb4bc16-60c9-49f1-a412-86f5b93694e9 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:34.807607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6cb4bc16-60c9-49f1-a412-86f5b93694e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:34.815456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1323/2617] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:34.819891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a1e9f-b499-4e05-955c-abef3ac63576 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:34.829584 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a1e9f-b499-4e05-955c-abef3ac63576 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:34.834474 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7e414-fb05-446d-8acd-a280763c12c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:34.835028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7e414-fb05-446d-8acd-a280763c12c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:34.835863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7e414-fb05-446d-8acd-a280763c12c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:34.836349 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7e414-fb05-446d-8acd-a280763c12c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:34.837279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7e414-fb05-446d-8acd-a280763c12c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:24:34.837594 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7e414-fb05-446d-8acd-a280763c12c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:34.839213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7e414-fb05-446d-8acd-a280763c12c9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:34.853138 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1295/2618] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:24:34.857042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4bc16-60c9-49f1-a412-86f5b93694e9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:34.866661 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4bc16-60c9-49f1-a412-86f5b93694e9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:34.871780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f8a6b0-d9c6-4167-a8e4-e9edbc221a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:34.872505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8a6b0-d9c6-4167-a8e4-e9edbc221a26 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:34.872858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8a6b0-d9c6-4167-a8e4-e9edbc221a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:34.873216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8a6b0-d9c6-4167-a8e4-e9edbc221a26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:34.873995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f8a6b0-d9c6-4167-a8e4-e9edbc221a26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:34.876002 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f8a6b0-d9c6-4167-a8e4-e9edbc221a26 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:34.999198 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1324/2619] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:34 2022] POST /identity/v3/projects => generated 387 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:35.003310 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7e414-fb05-446d-8acd-a280763c12c9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:35.014244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7e414-fb05-446d-8acd-a280763c12c9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:35.066113 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:35.072612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48c3b2-7858-40ad-ac7d-64340e745028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:35.073287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48c3b2-7858-40ad-ac7d-64340e745028 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:35.073578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48c3b2-7858-40ad-ac7d-64340e745028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:35.073854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48c3b2-7858-40ad-ac7d-64340e745028 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:35.074546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48c3b2-7858-40ad-ac7d-64340e745028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:35.076525 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48c3b2-7858-40ad-ac7d-64340e745028 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:35.164522 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1296/2620] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:35 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:35.168446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f8a6b0-d9c6-4167-a8e4-e9edbc221a26 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:35.179027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f8a6b0-d9c6-4167-a8e4-e9edbc221a26 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:35.183868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9e9e5-06ee-46c7-adc9-de6f8e353295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:35.184933 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9e9e5-06ee-46c7-adc9-de6f8e353295 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:35.185273 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9e9e5-06ee-46c7-adc9-de6f8e353295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:35.185554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9e9e5-06ee-46c7-adc9-de6f8e353295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:35.186528 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9e9e5-06ee-46c7-adc9-de6f8e353295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:35.188189 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9e9e5-06ee-46c7-adc9-de6f8e353295 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:35.201304 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1325/2621] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:35.205782 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48c3b2-7858-40ad-ac7d-64340e745028 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:35.219090 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48c3b2-7858-40ad-ac7d-64340e745028 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:35.224192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb111eb-96d0-4705-ba95-fc5270f2663a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:35.225056 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb111eb-96d0-4705-ba95-fc5270f2663a admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:35.225534 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb111eb-96d0-4705-ba95-fc5270f2663a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:35.226957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb111eb-96d0-4705-ba95-fc5270f2663a admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:35.228011 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb111eb-96d0-4705-ba95-fc5270f2663a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:35.229656 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb111eb-96d0-4705-ba95-fc5270f2663a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:35.240470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1297/2622] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:35.245695 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9e9e5-06ee-46c7-adc9-de6f8e353295 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:35.256392 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9e9e5-06ee-46c7-adc9-de6f8e353295 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:35.262667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8e261-0056-4aa7-a3eb-6bb70a1c8570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:35.263622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8e261-0056-4aa7-a3eb-6bb70a1c8570 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:35.263915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8e261-0056-4aa7-a3eb-6bb70a1c8570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:35.264187 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8e261-0056-4aa7-a3eb-6bb70a1c8570 admin admin] PATH_INFO: `/v3/projects/65fd152fa3a94aeba1d2f95fcfed1b96/users/9fb75b0324b1400c8732f35b48d14eb0/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:35.281242 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8e261-0056-4aa7-a3eb-6bb70a1c8570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65fd152fa3a94aeba1d2f95fcfed1b96, user_id=9fb75b0324b1400c8732f35b48d14eb0, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:35.283904 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8e261-0056-4aa7-a3eb-6bb70a1c8570 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:35.357866 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1326/2623] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:35 2022] PUT /identity/v3/projects/65fd152fa3a94aeba1d2f95fcfed1b96/users/9fb75b0324b1400c8732f35b48d14eb0/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:35.368788 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed61aab-3d20-42af-9540-ffdf28dea6a2 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:35.369077 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed61aab-3d20-42af-9540-ffdf28dea6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:35.369356 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed61aab-3d20-42af-9540-ffdf28dea6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:35.396523 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-eed61aab-3d20-42af-9540-ffdf28dea6a2 None None] MFA Rules not processed for user `9fb75b0324b1400c8732f35b48d14eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:35.397728 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-eed61aab-3d20-42af-9540-ffdf28dea6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:35.461466 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1298/2624] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:24:35.465927 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8e261-0056-4aa7-a3eb-6bb70a1c8570 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:35.504106 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:35.526794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8e261-0056-4aa7-a3eb-6bb70a1c8570 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:35.531509 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77171d9d-d24a-4b0a-9d56-6d136af1a8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:35.532165 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77171d9d-d24a-4b0a-9d56-6d136af1a8ab admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:35.532454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77171d9d-d24a-4b0a-9d56-6d136af1a8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:35.532733 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77171d9d-d24a-4b0a-9d56-6d136af1a8ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:35.533420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77171d9d-d24a-4b0a-9d56-6d136af1a8ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:35.535870 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77171d9d-d24a-4b0a-9d56-6d136af1a8ab admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:35.639270 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1327/2625] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:35 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:35.643231 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed61aab-3d20-42af-9540-ffdf28dea6a2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:35.658575 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed61aab-3d20-42af-9540-ffdf28dea6a2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:35.713627 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f958a-4ca9-4744-8a8a-d5b9caf5ab88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:35.714305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f958a-4ca9-4744-8a8a-d5b9caf5ab88 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:35.714593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f958a-4ca9-4744-8a8a-d5b9caf5ab88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:35.714873 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f958a-4ca9-4744-8a8a-d5b9caf5ab88 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:35.715611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f958a-4ca9-4744-8a8a-d5b9caf5ab88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:35.717532 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f958a-4ca9-4744-8a8a-d5b9caf5ab88 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:35.815161 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1299/2626] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:35 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:35.819240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77171d9d-d24a-4b0a-9d56-6d136af1a8ab admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:35.830068 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77171d9d-d24a-4b0a-9d56-6d136af1a8ab admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:35.834765 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e39cdd-15d5-4748-9049-83b881fc025d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:35.835465 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e39cdd-15d5-4748-9049-83b881fc025d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:35.835975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e39cdd-15d5-4748-9049-83b881fc025d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:35.836310 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e39cdd-15d5-4748-9049-83b881fc025d admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:35.837073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e39cdd-15d5-4748-9049-83b881fc025d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:35.838748 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e39cdd-15d5-4748-9049-83b881fc025d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:35.854530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1328/2627] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:35.855942 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f958a-4ca9-4744-8a8a-d5b9caf5ab88 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:35.867575 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f958a-4ca9-4744-8a8a-d5b9caf5ab88 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:35.872274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd679917-c0ea-445b-ba13-7bfc582deb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:35.873243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd679917-c0ea-445b-ba13-7bfc582deb61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:35.873528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd679917-c0ea-445b-ba13-7bfc582deb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:35.873809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd679917-c0ea-445b-ba13-7bfc582deb61 admin admin] PATH_INFO: `/v3/projects/9282831cf094486ca5671ba35997c8c9/users/e026619ed33e4acc856f8f27a24eac80/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:35.888743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd679917-c0ea-445b-ba13-7bfc582deb61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9282831cf094486ca5671ba35997c8c9, user_id=e026619ed33e4acc856f8f27a24eac80, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:35.891570 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd679917-c0ea-445b-ba13-7bfc582deb61 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:35.984231 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1300/2628] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:35 2022] PUT /identity/v3/projects/9282831cf094486ca5671ba35997c8c9/users/e026619ed33e4acc856f8f27a24eac80/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:35.988130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e39cdd-15d5-4748-9049-83b881fc025d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:35.998973 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e39cdd-15d5-4748-9049-83b881fc025d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:36.055911 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62b2bc-363b-4825-ab27-2356fd876c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:36.056642 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62b2bc-363b-4825-ab27-2356fd876c16 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:36.056925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62b2bc-363b-4825-ab27-2356fd876c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:36.057298 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62b2bc-363b-4825-ab27-2356fd876c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:36.058027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62b2bc-363b-4825-ab27-2356fd876c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:36.059684 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62b2bc-363b-4825-ab27-2356fd876c16 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:36.069497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1329/2629] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:36.074057 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd679917-c0ea-445b-ba13-7bfc582deb61 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:36.085229 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd679917-c0ea-445b-ba13-7bfc582deb61 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:36.090582 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce26aa-4490-412c-a1da-165779c5099a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:36.091508 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce26aa-4490-412c-a1da-165779c5099a admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:36.091792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce26aa-4490-412c-a1da-165779c5099a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:36.092068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce26aa-4490-412c-a1da-165779c5099a admin admin] PATH_INFO: `/v3/projects/9282831cf094486ca5671ba35997c8c9/users/e026619ed33e4acc856f8f27a24eac80/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:36.095665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce26aa-4490-412c-a1da-165779c5099a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9282831cf094486ca5671ba35997c8c9, user_id=e026619ed33e4acc856f8f27a24eac80, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:36.098333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce26aa-4490-412c-a1da-165779c5099a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:36.111302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1301/2630] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:36 2022] PUT /identity/v3/projects/9282831cf094486ca5671ba35997c8c9/users/e026619ed33e4acc856f8f27a24eac80/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:36.124504 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8be2b-cb55-447b-a087-a7b122a941d3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:36.124794 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8be2b-cb55-447b-a087-a7b122a941d3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:36.125234 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8be2b-cb55-447b-a087-a7b122a941d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:36.157976 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e8a8be2b-cb55-447b-a087-a7b122a941d3 None None] MFA Rules not processed for user `e026619ed33e4acc856f8f27a24eac80`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:36.159311 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e8a8be2b-cb55-447b-a087-a7b122a941d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:36.228685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1330/2631] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:36.242978 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce26aa-4490-412c-a1da-165779c5099a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:36.242978 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0cce26aa-4490-412c-a1da-165779c5099a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:36.264922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:36.309120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce26aa-4490-412c-a1da-165779c5099a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:36.313686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df53fe-8c65-471c-ac1b-0b71d42c2afc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:36.314343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df53fe-8c65-471c-ac1b-0b71d42c2afc service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:36.314627 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df53fe-8c65-471c-ac1b-0b71d42c2afc service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:36.314905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df53fe-8c65-471c-ac1b-0b71d42c2afc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:36.325435 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df53fe-8c65-471c-ac1b-0b71d42c2afc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:36.326596 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df53fe-8c65-471c-ac1b-0b71d42c2afc service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:36.336457 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1302/2632] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:36.469999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8be2b-cb55-447b-a087-a7b122a941d3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:36.506002 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:36.535295 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8be2b-cb55-447b-a087-a7b122a941d3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:36.542601 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e5a54-a305-48ea-9c50-d970f6163546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:36.543530 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e5a54-a305-48ea-9c50-d970f6163546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:36.543814 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e5a54-a305-48ea-9c50-d970f6163546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:36.544088 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e5a54-a305-48ea-9c50-d970f6163546 admin admin] PATH_INFO: `/v3/users/9fb75b0324b1400c8732f35b48d14eb0` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:36.545517 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e5a54-a305-48ea-9c50-d970f6163546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fb75b0324b1400c8732f35b48d14eb0)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:36.547585 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e5a54-a305-48ea-9c50-d970f6163546 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:36.729676 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f50e5a54-a305-48ea-9c50-d970f6163546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fb75b0324b1400c8732f35b48d14eb0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:36.743533 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f50e5a54-a305-48ea-9c50-d970f6163546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fb75b0324b1400c8732f35b48d14eb0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:36.834857 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f50e5a54-a305-48ea-9c50-d970f6163546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fb75b0324b1400c8732f35b48d14eb0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:36.836059 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f50e5a54-a305-48ea-9c50-d970f6163546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fb75b0324b1400c8732f35b48d14eb0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:36.843589 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f50e5a54-a305-48ea-9c50-d970f6163546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fb75b0324b1400c8732f35b48d14eb0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:36.854903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1331/2633] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:36 2022] DELETE /identity/v3/users/9fb75b0324b1400c8732f35b48d14eb0 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:36.859565 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51025974-b6ef-4ce7-b34d-7cc2577c2e55 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:36.859940 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51025974-b6ef-4ce7-b34d-7cc2577c2e55 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:36.860288 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51025974-b6ef-4ce7-b34d-7cc2577c2e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:36.877956 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-51025974-b6ef-4ce7-b34d-7cc2577c2e55 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:36.879280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-51025974-b6ef-4ce7-b34d-7cc2577c2e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:36.940658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1303/2634] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:36.950465 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e5a54-a305-48ea-9c50-d970f6163546 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:36.951939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f50e5a54-a305-48ea-9c50-d970f6163546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:37.014059 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e5a54-a305-48ea-9c50-d970f6163546 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:37.018818 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e856cc-ac0a-4844-a68b-3fda8e8f4c69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:37.019919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e856cc-ac0a-4844-a68b-3fda8e8f4c69 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:37.020207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e856cc-ac0a-4844-a68b-3fda8e8f4c69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:37.020675 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e856cc-ac0a-4844-a68b-3fda8e8f4c69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:37.031564 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e856cc-ac0a-4844-a68b-3fda8e8f4c69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:37.033805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e856cc-ac0a-4844-a68b-3fda8e8f4c69 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:37.045345 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1332/2635] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:37.569447 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51025974-b6ef-4ce7-b34d-7cc2577c2e55 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:37.571439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-51025974-b6ef-4ce7-b34d-7cc2577c2e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:37.590977 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51025974-b6ef-4ce7-b34d-7cc2577c2e55 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:37.598061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be24a82-e531-44b3-ad30-e843611d5c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:37.600177 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be24a82-e531-44b3-ad30-e843611d5c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:37.600177 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be24a82-e531-44b3-ad30-e843611d5c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:37.600404 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be24a82-e531-44b3-ad30-e843611d5c80 admin admin] PATH_INFO: `/v3/projects/65fd152fa3a94aeba1d2f95fcfed1b96` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:37.602343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be24a82-e531-44b3-ad30-e843611d5c80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65fd152fa3a94aeba1d2f95fcfed1b96)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:37.604559 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be24a82-e531-44b3-ad30-e843611d5c80 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:37.777406 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9be24a82-e531-44b3-ad30-e843611d5c80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65fd152fa3a94aeba1d2f95fcfed1b96', 'request_id': 'req-9be24a82-e531-44b3-ad30-e843611d5c80'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:37.784630 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9be24a82-e531-44b3-ad30-e843611d5c80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65fd152fa3a94aeba1d2f95fcfed1b96', 'request_id': 'req-9be24a82-e531-44b3-ad30-e843611d5c80'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:37.900779 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9be24a82-e531-44b3-ad30-e843611d5c80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65fd152fa3a94aeba1d2f95fcfed1b96', 'request_id': 'req-9be24a82-e531-44b3-ad30-e843611d5c80'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:37.913205 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9be24a82-e531-44b3-ad30-e843611d5c80 admin admin] The token cache is being invalidate because project 65fd152fa3a94aeba1d2f95fcfed1b96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:37.913506 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9be24a82-e531-44b3-ad30-e843611d5c80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:37.914449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9be24a82-e531-44b3-ad30-e843611d5c80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:37.916072 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1304/2636] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:37 2022] DELETE /identity/v3/projects/65fd152fa3a94aeba1d2f95fcfed1b96 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:37.920123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e856cc-ac0a-4844-a68b-3fda8e8f4c69 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:37.933635 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e856cc-ac0a-4844-a68b-3fda8e8f4c69 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:37.981107 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:37.989947 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb967f6d-e7c3-4cc4-97cf-79b5d574ca96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:37.990833 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb967f6d-e7c3-4cc4-97cf-79b5d574ca96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:37.991146 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb967f6d-e7c3-4cc4-97cf-79b5d574ca96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:37.991598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb967f6d-e7c3-4cc4-97cf-79b5d574ca96 admin admin] PATH_INFO: `/v3/users/e026619ed33e4acc856f8f27a24eac80` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:37.992779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb967f6d-e7c3-4cc4-97cf-79b5d574ca96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e026619ed33e4acc856f8f27a24eac80)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:37.994803 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb967f6d-e7c3-4cc4-97cf-79b5d574ca96 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:38.275004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bb967f6d-e7c3-4cc4-97cf-79b5d574ca96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e026619ed33e4acc856f8f27a24eac80'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:38.286843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bb967f6d-e7c3-4cc4-97cf-79b5d574ca96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e026619ed33e4acc856f8f27a24eac80'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:38.394482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bb967f6d-e7c3-4cc4-97cf-79b5d574ca96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e026619ed33e4acc856f8f27a24eac80'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:38.395114 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bb967f6d-e7c3-4cc4-97cf-79b5d574ca96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e026619ed33e4acc856f8f27a24eac80'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:38.404211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bb967f6d-e7c3-4cc4-97cf-79b5d574ca96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e026619ed33e4acc856f8f27a24eac80'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:38.410395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1333/2637] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:37 2022] DELETE /identity/v3/users/e026619ed33e4acc856f8f27a24eac80 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:39.230214 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be24a82-e531-44b3-ad30-e843611d5c80 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:39.237654 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9be24a82-e531-44b3-ad30-e843611d5c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:39.245211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be24a82-e531-44b3-ad30-e843611d5c80 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:39.295927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:39.312631 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f08a4-f7c9-4de5-a1a2-8ce9f8e0eab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:39.315545 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f08a4-f7c9-4de5-a1a2-8ce9f8e0eab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:39.315545 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f08a4-f7c9-4de5-a1a2-8ce9f8e0eab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:39.315545 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f08a4-f7c9-4de5-a1a2-8ce9f8e0eab4 admin admin] PATH_INFO: `/v3/projects/9282831cf094486ca5671ba35997c8c9` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:39.315545 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f08a4-f7c9-4de5-a1a2-8ce9f8e0eab4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9282831cf094486ca5671ba35997c8c9)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:39.317533 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f08a4-f7c9-4de5-a1a2-8ce9f8e0eab4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:39.437723 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-377f08a4-f7c9-4de5-a1a2-8ce9f8e0eab4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9282831cf094486ca5671ba35997c8c9', 'request_id': 'req-377f08a4-f7c9-4de5-a1a2-8ce9f8e0eab4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:39.443875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-377f08a4-f7c9-4de5-a1a2-8ce9f8e0eab4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9282831cf094486ca5671ba35997c8c9', 'request_id': 'req-377f08a4-f7c9-4de5-a1a2-8ce9f8e0eab4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:39.563453 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-377f08a4-f7c9-4de5-a1a2-8ce9f8e0eab4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9282831cf094486ca5671ba35997c8c9', 'request_id': 'req-377f08a4-f7c9-4de5-a1a2-8ce9f8e0eab4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:39.575375 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-377f08a4-f7c9-4de5-a1a2-8ce9f8e0eab4 admin admin] The token cache is being invalidate because project 9282831cf094486ca5671ba35997c8c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:39.575695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-377f08a4-f7c9-4de5-a1a2-8ce9f8e0eab4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:39.576321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-377f08a4-f7c9-4de5-a1a2-8ce9f8e0eab4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:39.577975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1305/2638] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:39 2022] DELETE /identity/v3/projects/9282831cf094486ca5671ba35997c8c9 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:39.582736 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e6ddd-c121-4faf-bee0-81ea53559ff3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:39.583194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e6ddd-c121-4faf-bee0-81ea53559ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:39.583403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e6ddd-c121-4faf-bee0-81ea53559ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:39.602778 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fc9e6ddd-c121-4faf-bee0-81ea53559ff3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:39.604027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fc9e6ddd-c121-4faf-bee0-81ea53559ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:39.666382 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1334/2639] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:39.674279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb7273-9a1c-464e-af58-98d1ca7e45a0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:39.674572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb7273-9a1c-464e-af58-98d1ca7e45a0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:39.674854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb7273-9a1c-464e-af58-98d1ca7e45a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:39.693088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7dcb7273-9a1c-464e-af58-98d1ca7e45a0 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:39.694467 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7dcb7273-9a1c-464e-af58-98d1ca7e45a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:39.703196 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1306/2640] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:24:39.707386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e6ddd-c121-4faf-bee0-81ea53559ff3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:39.718212 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e6ddd-c121-4faf-bee0-81ea53559ff3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:39.722986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2229c-ca93-404c-841c-99bb5c74b20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:39.723827 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2229c-ca93-404c-841c-99bb5c74b20d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:39.724108 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2229c-ca93-404c-841c-99bb5c74b20d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:39.724388 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2229c-ca93-404c-841c-99bb5c74b20d admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:39.725105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2229c-ca93-404c-841c-99bb5c74b20d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:24:39.725413 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2229c-ca93-404c-841c-99bb5c74b20d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:39.727313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2229c-ca93-404c-841c-99bb5c74b20d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:39.739156 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1335/2641] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:24:39.744067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb7273-9a1c-464e-af58-98d1ca7e45a0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:39.753212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb7273-9a1c-464e-af58-98d1ca7e45a0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:39.758651 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578033a-50d8-4921-9c42-193b80404c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:39.760013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578033a-50d8-4921-9c42-193b80404c8e admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:39.760013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578033a-50d8-4921-9c42-193b80404c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:39.760013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578033a-50d8-4921-9c42-193b80404c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:39.760567 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578033a-50d8-4921-9c42-193b80404c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:39.762448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578033a-50d8-4921-9c42-193b80404c8e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:39.865709 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1307/2642] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:39 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:39.870432 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2229c-ca93-404c-841c-99bb5c74b20d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:39.880629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2229c-ca93-404c-841c-99bb5c74b20d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:39.942115 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95239b5-b362-4d7a-80aa-a43e0e91f1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:39.943007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95239b5-b362-4d7a-80aa-a43e0e91f1c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:39.943347 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95239b5-b362-4d7a-80aa-a43e0e91f1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:39.943704 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95239b5-b362-4d7a-80aa-a43e0e91f1c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:39.944398 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95239b5-b362-4d7a-80aa-a43e0e91f1c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:39.947059 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95239b5-b362-4d7a-80aa-a43e0e91f1c5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:39.954301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:40.026883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1336/2643] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:39 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:40.031319 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578033a-50d8-4921-9c42-193b80404c8e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:40.042026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578033a-50d8-4921-9c42-193b80404c8e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:40.052490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbbb57-e67f-40d8-bf3d-20c449b4d36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:40.053249 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbbb57-e67f-40d8-bf3d-20c449b4d36c admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:40.053538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbbb57-e67f-40d8-bf3d-20c449b4d36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:40.053836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbbb57-e67f-40d8-bf3d-20c449b4d36c admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:40.055163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbbb57-e67f-40d8-bf3d-20c449b4d36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:40.060907 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbbb57-e67f-40d8-bf3d-20c449b4d36c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:40.075457 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1308/2644] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:40.076823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95239b5-b362-4d7a-80aa-a43e0e91f1c5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:40.086914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95239b5-b362-4d7a-80aa-a43e0e91f1c5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:40.093628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a53f78-cff5-4ff0-9c1d-37ff24c5c765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:40.094322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a53f78-cff5-4ff0-9c1d-37ff24c5c765 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:40.094604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a53f78-cff5-4ff0-9c1d-37ff24c5c765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:40.094877 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a53f78-cff5-4ff0-9c1d-37ff24c5c765 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:40.095637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a53f78-cff5-4ff0-9c1d-37ff24c5c765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:40.097217 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a53f78-cff5-4ff0-9c1d-37ff24c5c765 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:40.111148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1337/2645] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:40.115266 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbbb57-e67f-40d8-bf3d-20c449b4d36c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:40.126339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbbb57-e67f-40d8-bf3d-20c449b4d36c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:40.131054 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43196b71-acaf-432f-849a-3a0d72a916fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:40.131986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43196b71-acaf-432f-849a-3a0d72a916fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:40.132482 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43196b71-acaf-432f-849a-3a0d72a916fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:40.133052 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43196b71-acaf-432f-849a-3a0d72a916fb admin admin] PATH_INFO: `/v3/projects/6b229777a91f466ebc4dc84d90186841/users/a1b124eac1374fc2b2c5224390e77622/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:40.148006 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43196b71-acaf-432f-849a-3a0d72a916fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b229777a91f466ebc4dc84d90186841, user_id=a1b124eac1374fc2b2c5224390e77622, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:40.151303 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43196b71-acaf-432f-849a-3a0d72a916fb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:40.252454 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1309/2646] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:40 2022] PUT /identity/v3/projects/6b229777a91f466ebc4dc84d90186841/users/a1b124eac1374fc2b2c5224390e77622/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:40.264149 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f43cb6-82cb-4d36-afeb-f465d8180c41 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:40.264439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f43cb6-82cb-4d36-afeb-f465d8180c41 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:40.264720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f43cb6-82cb-4d36-afeb-f465d8180c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:40.292452 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-27f43cb6-82cb-4d36-afeb-f465d8180c41 None None] MFA Rules not processed for user `a1b124eac1374fc2b2c5224390e77622`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:40.293680 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-27f43cb6-82cb-4d36-afeb-f465d8180c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:40.356489 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1338/2647] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:40.361065 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43196b71-acaf-432f-849a-3a0d72a916fb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:40.423662 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43196b71-acaf-432f-849a-3a0d72a916fb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:40.428372 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c974b67-6110-40b7-8448-72eee4ebbcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:40.429605 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c974b67-6110-40b7-8448-72eee4ebbcf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:40.430020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c974b67-6110-40b7-8448-72eee4ebbcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:40.430416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c974b67-6110-40b7-8448-72eee4ebbcf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:40.431122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c974b67-6110-40b7-8448-72eee4ebbcf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:40.433121 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c974b67-6110-40b7-8448-72eee4ebbcf2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:40.446941 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:40.539964 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1310/2648] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:40 2022] POST /identity/v3/projects => generated 381 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:40.543975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f43cb6-82cb-4d36-afeb-f465d8180c41 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:40.554987 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f43cb6-82cb-4d36-afeb-f465d8180c41 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:40.610493 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9065f20-f215-4f55-84bc-c91c30b45f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:40.611197 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9065f20-f215-4f55-84bc-c91c30b45f38 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:40.611432 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9065f20-f215-4f55-84bc-c91c30b45f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:40.611713 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9065f20-f215-4f55-84bc-c91c30b45f38 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:40.612412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9065f20-f215-4f55-84bc-c91c30b45f38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:40.614330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9065f20-f215-4f55-84bc-c91c30b45f38 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:40.688864 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1339/2649] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:40 2022] POST /identity/v3/users => generated 297 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:40.693041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c974b67-6110-40b7-8448-72eee4ebbcf2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:40.703609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c974b67-6110-40b7-8448-72eee4ebbcf2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:40.708233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c14d0c-1d4b-4177-aaf9-b663f3b1b78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:40.708893 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c14d0c-1d4b-4177-aaf9-b663f3b1b78b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:40.709181 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c14d0c-1d4b-4177-aaf9-b663f3b1b78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:40.709461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c14d0c-1d4b-4177-aaf9-b663f3b1b78b admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:40.710725 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c14d0c-1d4b-4177-aaf9-b663f3b1b78b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:40.712537 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c14d0c-1d4b-4177-aaf9-b663f3b1b78b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:40.723351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1311/2650] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:40.729534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9065f20-f215-4f55-84bc-c91c30b45f38 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:40.739761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9065f20-f215-4f55-84bc-c91c30b45f38 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:40.744771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b0075-4d5f-4653-829f-a228e8ffd912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:40.745655 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b0075-4d5f-4653-829f-a228e8ffd912 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:40.745936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b0075-4d5f-4653-829f-a228e8ffd912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:40.746211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b0075-4d5f-4653-829f-a228e8ffd912 admin admin] PATH_INFO: `/v3/projects/26ed7f6c6738410d840612675cb3635d/users/05aa935fafb0449fadae9e97ad39675a/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:40.761017 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b0075-4d5f-4653-829f-a228e8ffd912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26ed7f6c6738410d840612675cb3635d, user_id=05aa935fafb0449fadae9e97ad39675a, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:40.763734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b0075-4d5f-4653-829f-a228e8ffd912 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:40.831788 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1340/2651] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:40 2022] PUT /identity/v3/projects/26ed7f6c6738410d840612675cb3635d/users/05aa935fafb0449fadae9e97ad39675a/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:40.836290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c14d0c-1d4b-4177-aaf9-b663f3b1b78b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:40.846270 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c14d0c-1d4b-4177-aaf9-b663f3b1b78b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:40.903796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18edcb59-f9c1-408a-82f8-c5586cde16f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:40.904468 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edcb59-f9c1-408a-82f8-c5586cde16f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:40.904752 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edcb59-f9c1-408a-82f8-c5586cde16f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:40.905028 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edcb59-f9c1-408a-82f8-c5586cde16f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:40.905747 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18edcb59-f9c1-408a-82f8-c5586cde16f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:40.907360 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18edcb59-f9c1-408a-82f8-c5586cde16f2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:40.917604 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1312/2652] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:40.922571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b0075-4d5f-4653-829f-a228e8ffd912 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:40.933329 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b0075-4d5f-4653-829f-a228e8ffd912 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:40.938321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece07e63-1bd4-46ae-899f-d1030cca8775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:40.939251 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece07e63-1bd4-46ae-899f-d1030cca8775 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:40.939539 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece07e63-1bd4-46ae-899f-d1030cca8775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:40.939858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece07e63-1bd4-46ae-899f-d1030cca8775 admin admin] PATH_INFO: `/v3/projects/26ed7f6c6738410d840612675cb3635d/users/05aa935fafb0449fadae9e97ad39675a/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:40.941826 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece07e63-1bd4-46ae-899f-d1030cca8775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26ed7f6c6738410d840612675cb3635d, user_id=05aa935fafb0449fadae9e97ad39675a, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:40.945040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece07e63-1bd4-46ae-899f-d1030cca8775 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:40.955902 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1341/2653] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:40 2022] PUT /identity/v3/projects/26ed7f6c6738410d840612675cb3635d/users/05aa935fafb0449fadae9e97ad39675a/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:40.966996 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e5f58-a476-4ac3-bef2-cd6c766069b5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:40.967326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e5f58-a476-4ac3-bef2-cd6c766069b5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:40.967635 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e5f58-a476-4ac3-bef2-cd6c766069b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:40.994210 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-087e5f58-a476-4ac3-bef2-cd6c766069b5 None None] MFA Rules not processed for user `05aa935fafb0449fadae9e97ad39675a`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:40.995561 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-087e5f58-a476-4ac3-bef2-cd6c766069b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:41.068597 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1313/2654] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:41.081061 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece07e63-1bd4-46ae-899f-d1030cca8775 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:41.083069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ece07e63-1bd4-46ae-899f-d1030cca8775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:41.131001 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:41.173963 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece07e63-1bd4-46ae-899f-d1030cca8775 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:41.178111 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d06174-8b0a-4163-aa2c-4b7c59d69332 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:41.178891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d06174-8b0a-4163-aa2c-4b7c59d69332 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:41.179491 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d06174-8b0a-4163-aa2c-4b7c59d69332 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:41.180313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d06174-8b0a-4163-aa2c-4b7c59d69332 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:41.192538 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d06174-8b0a-4163-aa2c-4b7c59d69332 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:41.194107 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d06174-8b0a-4163-aa2c-4b7c59d69332 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:41.207678 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1342/2655] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:41.344886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e5f58-a476-4ac3-bef2-cd6c766069b5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:41.357966 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e5f58-a476-4ac3-bef2-cd6c766069b5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:41.360944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee25338-b7c7-4f24-b839-0dfc8b32ac70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:41.361685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee25338-b7c7-4f24-b839-0dfc8b32ac70 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:41.361981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee25338-b7c7-4f24-b839-0dfc8b32ac70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:41.362263 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee25338-b7c7-4f24-b839-0dfc8b32ac70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:41.421463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee25338-b7c7-4f24-b839-0dfc8b32ac70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:41.422918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee25338-b7c7-4f24-b839-0dfc8b32ac70 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:41.432302 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1314/2656] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:41.915407 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d06174-8b0a-4163-aa2c-4b7c59d69332 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:41.976458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d06174-8b0a-4163-aa2c-4b7c59d69332 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:41.980954 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2f1d6-10c4-4391-9ab9-5a4023b987b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:41.981822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2f1d6-10c4-4391-9ab9-5a4023b987b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:41.982109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2f1d6-10c4-4391-9ab9-5a4023b987b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:41.982388 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2f1d6-10c4-4391-9ab9-5a4023b987b8 admin admin] PATH_INFO: `/v3/users/a1b124eac1374fc2b2c5224390e77622` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:41.983596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2f1d6-10c4-4391-9ab9-5a4023b987b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1b124eac1374fc2b2c5224390e77622)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:41.985531 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2f1d6-10c4-4391-9ab9-5a4023b987b8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:42.147246 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4a2f1d6-10c4-4391-9ab9-5a4023b987b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1b124eac1374fc2b2c5224390e77622'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:42.233157 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4a2f1d6-10c4-4391-9ab9-5a4023b987b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1b124eac1374fc2b2c5224390e77622'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:42.280064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4a2f1d6-10c4-4391-9ab9-5a4023b987b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1b124eac1374fc2b2c5224390e77622'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:42.284216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4a2f1d6-10c4-4391-9ab9-5a4023b987b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1b124eac1374fc2b2c5224390e77622'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:42.288563 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4a2f1d6-10c4-4391-9ab9-5a4023b987b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1b124eac1374fc2b2c5224390e77622'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:42.294766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1343/2657] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:41 2022] DELETE /identity/v3/users/a1b124eac1374fc2b2c5224390e77622 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:42.299490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf988ba-43c7-4b96-8938-d63f651d8d56 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:42.299786 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf988ba-43c7-4b96-8938-d63f651d8d56 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:42.300066 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf988ba-43c7-4b96-8938-d63f651d8d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:42.307363 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:42.318375 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2bf988ba-43c7-4b96-8938-d63f651d8d56 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:42.320439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2bf988ba-43c7-4b96-8938-d63f651d8d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:42.387270 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1315/2658] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:42.393639 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2f1d6-10c4-4391-9ab9-5a4023b987b8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:42.395098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c4a2f1d6-10c4-4391-9ab9-5a4023b987b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:42.442902 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:42.454403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2f1d6-10c4-4391-9ab9-5a4023b987b8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:42.458335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7299c1f-b718-43b3-bfc6-599be4bd9e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:42.458980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7299c1f-b718-43b3-bfc6-599be4bd9e97 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:42.459296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7299c1f-b718-43b3-bfc6-599be4bd9e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:42.459570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7299c1f-b718-43b3-bfc6-599be4bd9e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:42.469360 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7299c1f-b718-43b3-bfc6-599be4bd9e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:42.470808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7299c1f-b718-43b3-bfc6-599be4bd9e97 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:42.482617 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1344/2659] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:42.904374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf988ba-43c7-4b96-8938-d63f651d8d56 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:42.904374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2bf988ba-43c7-4b96-8938-d63f651d8d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:42.917225 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf988ba-43c7-4b96-8938-d63f651d8d56 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:42.926688 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e86755-0cbc-4c5d-835d-2387d886529b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:42.927578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e86755-0cbc-4c5d-835d-2387d886529b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:42.927956 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e86755-0cbc-4c5d-835d-2387d886529b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:42.928189 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e86755-0cbc-4c5d-835d-2387d886529b admin admin] PATH_INFO: `/v3/projects/6b229777a91f466ebc4dc84d90186841` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:42.929424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e86755-0cbc-4c5d-835d-2387d886529b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b229777a91f466ebc4dc84d90186841)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:42.931451 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e86755-0cbc-4c5d-835d-2387d886529b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:43.028837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d5e86755-0cbc-4c5d-835d-2387d886529b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b229777a91f466ebc4dc84d90186841', 'request_id': 'req-d5e86755-0cbc-4c5d-835d-2387d886529b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:43.034945 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d5e86755-0cbc-4c5d-835d-2387d886529b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b229777a91f466ebc4dc84d90186841', 'request_id': 'req-d5e86755-0cbc-4c5d-835d-2387d886529b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:43.108740 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d5e86755-0cbc-4c5d-835d-2387d886529b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b229777a91f466ebc4dc84d90186841', 'request_id': 'req-d5e86755-0cbc-4c5d-835d-2387d886529b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:43.119645 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d5e86755-0cbc-4c5d-835d-2387d886529b admin admin] The token cache is being invalidate because project 6b229777a91f466ebc4dc84d90186841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:43.120147 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d5e86755-0cbc-4c5d-835d-2387d886529b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:43.120664 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d5e86755-0cbc-4c5d-835d-2387d886529b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:43.122218 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1316/2660] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:42 2022] DELETE /identity/v3/projects/6b229777a91f466ebc4dc84d90186841 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:43.126293 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7299c1f-b718-43b3-bfc6-599be4bd9e97 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:43.136396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7299c1f-b718-43b3-bfc6-599be4bd9e97 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:43.193151 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4556532-200c-492a-878a-d9903a491ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:43.193949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4556532-200c-492a-878a-d9903a491ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:43.194234 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4556532-200c-492a-878a-d9903a491ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:43.194511 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4556532-200c-492a-878a-d9903a491ef9 admin admin] PATH_INFO: `/v3/users/05aa935fafb0449fadae9e97ad39675a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:43.195754 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4556532-200c-492a-878a-d9903a491ef9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05aa935fafb0449fadae9e97ad39675a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:43.197656 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4556532-200c-492a-878a-d9903a491ef9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:43.348221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4556532-200c-492a-878a-d9903a491ef9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05aa935fafb0449fadae9e97ad39675a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:43.360522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4556532-200c-492a-878a-d9903a491ef9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05aa935fafb0449fadae9e97ad39675a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:43.446472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4556532-200c-492a-878a-d9903a491ef9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05aa935fafb0449fadae9e97ad39675a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:43.447069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4556532-200c-492a-878a-d9903a491ef9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05aa935fafb0449fadae9e97ad39675a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:43.454441 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c4556532-200c-492a-878a-d9903a491ef9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05aa935fafb0449fadae9e97ad39675a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:43.460529 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1345/2661] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:43 2022] DELETE /identity/v3/users/05aa935fafb0449fadae9e97ad39675a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:43.892886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e86755-0cbc-4c5d-835d-2387d886529b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:43.894324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d5e86755-0cbc-4c5d-835d-2387d886529b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:43.907644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e86755-0cbc-4c5d-835d-2387d886529b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:43.972652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f4554-1b83-494e-9c65-a3b53b14d11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:43.973438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f4554-1b83-494e-9c65-a3b53b14d11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:43.973720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f4554-1b83-494e-9c65-a3b53b14d11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:43.973994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f4554-1b83-494e-9c65-a3b53b14d11b admin admin] PATH_INFO: `/v3/projects/26ed7f6c6738410d840612675cb3635d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:43.975165 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f4554-1b83-494e-9c65-a3b53b14d11b admin admin] RBAC: Authorizing `identity:delete_project(project_id=26ed7f6c6738410d840612675cb3635d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:43.977119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f4554-1b83-494e-9c65-a3b53b14d11b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:44.061936 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:44.075278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad5f4554-1b83-494e-9c65-a3b53b14d11b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26ed7f6c6738410d840612675cb3635d', 'request_id': 'req-ad5f4554-1b83-494e-9c65-a3b53b14d11b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:44.081440 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad5f4554-1b83-494e-9c65-a3b53b14d11b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26ed7f6c6738410d840612675cb3635d', 'request_id': 'req-ad5f4554-1b83-494e-9c65-a3b53b14d11b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:44.157531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad5f4554-1b83-494e-9c65-a3b53b14d11b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26ed7f6c6738410d840612675cb3635d', 'request_id': 'req-ad5f4554-1b83-494e-9c65-a3b53b14d11b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:44.168914 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad5f4554-1b83-494e-9c65-a3b53b14d11b admin admin] The token cache is being invalidate because project 26ed7f6c6738410d840612675cb3635d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:44.169233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad5f4554-1b83-494e-9c65-a3b53b14d11b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:44.169871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad5f4554-1b83-494e-9c65-a3b53b14d11b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:44.171460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1317/2662] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:43 2022] DELETE /identity/v3/projects/26ed7f6c6738410d840612675cb3635d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:44.177079 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5083af-bd4a-4a2b-8155-826aa48af2ec None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:44.177564 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5083af-bd4a-4a2b-8155-826aa48af2ec None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:44.177673 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5083af-bd4a-4a2b-8155-826aa48af2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:44.197444 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7b5083af-bd4a-4a2b-8155-826aa48af2ec None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:44.199211 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7b5083af-bd4a-4a2b-8155-826aa48af2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:44.257697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1346/2663] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:44.265465 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c2dad-d57d-48a2-b94a-a32625976fc1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:44.265760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c2dad-d57d-48a2-b94a-a32625976fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:44.266045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c2dad-d57d-48a2-b94a-a32625976fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:44.285082 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6f1c2dad-d57d-48a2-b94a-a32625976fc1 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:44.286286 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6f1c2dad-d57d-48a2-b94a-a32625976fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:44.294993 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1318/2664] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:44.299112 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5083af-bd4a-4a2b-8155-826aa48af2ec None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:44.305845 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:44.309706 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5083af-bd4a-4a2b-8155-826aa48af2ec None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:44.314454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93386dbc-cf3f-4a20-b84d-e6ad56b0fdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:44.319445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93386dbc-cf3f-4a20-b84d-e6ad56b0fdd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:44.319445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93386dbc-cf3f-4a20-b84d-e6ad56b0fdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:44.319445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93386dbc-cf3f-4a20-b84d-e6ad56b0fdd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:44.319445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93386dbc-cf3f-4a20-b84d-e6ad56b0fdd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:24:44.319445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93386dbc-cf3f-4a20-b84d-e6ad56b0fdd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:44.319445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93386dbc-cf3f-4a20-b84d-e6ad56b0fdd9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:44.329600 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1347/2665] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:24:44.333735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c2dad-d57d-48a2-b94a-a32625976fc1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:44.343135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c2dad-d57d-48a2-b94a-a32625976fc1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:44.347947 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b240d6a-27d3-45b2-9a33-085ff0fb6ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:44.350517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b240d6a-27d3-45b2-9a33-085ff0fb6ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:44.350517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b240d6a-27d3-45b2-9a33-085ff0fb6ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:44.350517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b240d6a-27d3-45b2-9a33-085ff0fb6ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:44.350517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b240d6a-27d3-45b2-9a33-085ff0fb6ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:44.352449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b240d6a-27d3-45b2-9a33-085ff0fb6ab1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:44.449938 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1319/2666] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:44 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:44.454351 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93386dbc-cf3f-4a20-b84d-e6ad56b0fdd9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:44.464488 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93386dbc-cf3f-4a20-b84d-e6ad56b0fdd9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:44.520361 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e66513-a627-4147-972f-43a7632482c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:44.520952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e66513-a627-4147-972f-43a7632482c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:44.521236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e66513-a627-4147-972f-43a7632482c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:44.521512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e66513-a627-4147-972f-43a7632482c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:44.522190 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e66513-a627-4147-972f-43a7632482c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:44.524374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e66513-a627-4147-972f-43a7632482c6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:44.617412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1348/2667] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:44 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:44.621729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b240d6a-27d3-45b2-9a33-085ff0fb6ab1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:44.632368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b240d6a-27d3-45b2-9a33-085ff0fb6ab1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:44.637232 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e045d25-e001-4c55-bfb7-69d410e6fc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:44.637912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e045d25-e001-4c55-bfb7-69d410e6fc57 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:44.638176 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e045d25-e001-4c55-bfb7-69d410e6fc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:44.638452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e045d25-e001-4c55-bfb7-69d410e6fc57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:44.639171 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e045d25-e001-4c55-bfb7-69d410e6fc57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:44.640781 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e045d25-e001-4c55-bfb7-69d410e6fc57 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:44.652239 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1320/2668] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:44.656106 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e66513-a627-4147-972f-43a7632482c6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:44.667142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e66513-a627-4147-972f-43a7632482c6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:44.671862 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7acac-b6ef-4aee-b882-b2389b881bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:44.672527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7acac-b6ef-4aee-b882-b2389b881bfa admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:44.672810 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7acac-b6ef-4aee-b882-b2389b881bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:44.673089 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7acac-b6ef-4aee-b882-b2389b881bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:44.673816 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7acac-b6ef-4aee-b882-b2389b881bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:44.675452 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7acac-b6ef-4aee-b882-b2389b881bfa admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:44.685469 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1349/2669] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:44.689468 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e045d25-e001-4c55-bfb7-69d410e6fc57 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:44.699569 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e045d25-e001-4c55-bfb7-69d410e6fc57 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:44.704247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98be6a-a6e1-467f-8759-4f170930715e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:44.705119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98be6a-a6e1-467f-8759-4f170930715e admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:44.705403 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98be6a-a6e1-467f-8759-4f170930715e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:44.705685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98be6a-a6e1-467f-8759-4f170930715e admin admin] PATH_INFO: `/v3/projects/b9bcc2607bca49adb438229d27b252c8/users/be34834fdcc84766aaddc16cfbff2c2d/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:44.720815 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98be6a-a6e1-467f-8759-4f170930715e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9bcc2607bca49adb438229d27b252c8, user_id=be34834fdcc84766aaddc16cfbff2c2d, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:44.723500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98be6a-a6e1-467f-8759-4f170930715e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:44.797423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1321/2670] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:44 2022] PUT /identity/v3/projects/b9bcc2607bca49adb438229d27b252c8/users/be34834fdcc84766aaddc16cfbff2c2d/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:44.812710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebd25a-e9c6-4c24-99c3-ed025dc2c705 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:44.812710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebd25a-e9c6-4c24-99c3-ed025dc2c705 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:44.812710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebd25a-e9c6-4c24-99c3-ed025dc2c705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:44.839682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c9ebd25a-e9c6-4c24-99c3-ed025dc2c705 None None] MFA Rules not processed for user `be34834fdcc84766aaddc16cfbff2c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:44.841124 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c9ebd25a-e9c6-4c24-99c3-ed025dc2c705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:44.911929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1350/2671] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:44.923953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98be6a-a6e1-467f-8759-4f170930715e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:44.925643 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bc98be6a-a6e1-467f-8759-4f170930715e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:44.986479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98be6a-a6e1-467f-8759-4f170930715e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:44.990743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d73fc-5104-44fb-a1ec-34eefe5f065e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:44.991510 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d73fc-5104-44fb-a1ec-34eefe5f065e service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:44.992421 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d73fc-5104-44fb-a1ec-34eefe5f065e service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:44.992804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d73fc-5104-44fb-a1ec-34eefe5f065e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:45.002140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d73fc-5104-44fb-a1ec-34eefe5f065e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:45.003597 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d73fc-5104-44fb-a1ec-34eefe5f065e service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:45.013622 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1322/2672] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:45.034243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebd25a-e9c6-4c24-99c3-ed025dc2c705 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:45.103610 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebd25a-e9c6-4c24-99c3-ed025dc2c705 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:45.108478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9b884-7e22-402a-800d-660470d5ce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:45.109473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9b884-7e22-402a-800d-660470d5ce7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:45.109756 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9b884-7e22-402a-800d-660470d5ce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:45.110020 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9b884-7e22-402a-800d-660470d5ce7a admin admin] PATH_INFO: `/v3/users/be34834fdcc84766aaddc16cfbff2c2d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:45.111386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9b884-7e22-402a-800d-660470d5ce7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=be34834fdcc84766aaddc16cfbff2c2d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:45.113296 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9b884-7e22-402a-800d-660470d5ce7a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:45.114984 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:45.327718 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-afa9b884-7e22-402a-800d-660470d5ce7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be34834fdcc84766aaddc16cfbff2c2d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:45.340429 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-afa9b884-7e22-402a-800d-660470d5ce7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be34834fdcc84766aaddc16cfbff2c2d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:45.434570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-afa9b884-7e22-402a-800d-660470d5ce7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be34834fdcc84766aaddc16cfbff2c2d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:45.435206 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-afa9b884-7e22-402a-800d-660470d5ce7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be34834fdcc84766aaddc16cfbff2c2d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:45.442907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-afa9b884-7e22-402a-800d-660470d5ce7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be34834fdcc84766aaddc16cfbff2c2d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:45.450365 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1351/2673] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:45 2022] DELETE /identity/v3/users/be34834fdcc84766aaddc16cfbff2c2d => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:45.454950 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403d3c4-8dfe-4fe1-adc1-1766a1d0cbb3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:45.455272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403d3c4-8dfe-4fe1-adc1-1766a1d0cbb3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:45.455556 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403d3c4-8dfe-4fe1-adc1-1766a1d0cbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:45.473080 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3403d3c4-8dfe-4fe1-adc1-1766a1d0cbb3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:45.474326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3403d3c4-8dfe-4fe1-adc1-1766a1d0cbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:45.526812 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:45.536636 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1323/2674] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:24:45.547748 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9b884-7e22-402a-800d-660470d5ce7a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:45.549571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-afa9b884-7e22-402a-800d-660470d5ce7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:45.599462 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403d3c4-8dfe-4fe1-adc1-1766a1d0cbb3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:45.600985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3403d3c4-8dfe-4fe1-adc1-1766a1d0cbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:45.654983 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9b884-7e22-402a-800d-660470d5ce7a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:45.664383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b5372-4221-4d3b-b579-6c7f62e336bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:45.666343 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b5372-4221-4d3b-b579-6c7f62e336bd service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:45.666653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b5372-4221-4d3b-b579-6c7f62e336bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:45.666937 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b5372-4221-4d3b-b579-6c7f62e336bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:45.691108 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b5372-4221-4d3b-b579-6c7f62e336bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:45.692626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b5372-4221-4d3b-b579-6c7f62e336bd service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:45.716921 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1352/2675] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:45.720712 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403d3c4-8dfe-4fe1-adc1-1766a1d0cbb3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:45.728045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18091c81-1cc6-4803-80c8-359ae5b6b4a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:45.729024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18091c81-1cc6-4803-80c8-359ae5b6b4a5 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:45.729429 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18091c81-1cc6-4803-80c8-359ae5b6b4a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:45.729830 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18091c81-1cc6-4803-80c8-359ae5b6b4a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:45.732115 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-18091c81-1cc6-4803-80c8-359ae5b6b4a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:45.749455 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18091c81-1cc6-4803-80c8-359ae5b6b4a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:45.751236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18091c81-1cc6-4803-80c8-359ae5b6b4a5 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:45.772301 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1324/2676] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:24:46.471254 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b5372-4221-4d3b-b579-6c7f62e336bd service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:46.473564 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c94b5372-4221-4d3b-b579-6c7f62e336bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:46.487433 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b5372-4221-4d3b-b579-6c7f62e336bd service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:46.494456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4ca85-4ff1-451d-8750-3ddf03d6d4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:46.495369 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4ca85-4ff1-451d-8750-3ddf03d6d4ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:46.495653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4ca85-4ff1-451d-8750-3ddf03d6d4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:46.495912 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4ca85-4ff1-451d-8750-3ddf03d6d4ed admin admin] PATH_INFO: `/v3/projects/b9bcc2607bca49adb438229d27b252c8` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:46.497418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4ca85-4ff1-451d-8750-3ddf03d6d4ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9bcc2607bca49adb438229d27b252c8)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:46.505191 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4ca85-4ff1-451d-8750-3ddf03d6d4ed admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:46.636342 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-09e4ca85-4ff1-451d-8750-3ddf03d6d4ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9bcc2607bca49adb438229d27b252c8', 'request_id': 'req-09e4ca85-4ff1-451d-8750-3ddf03d6d4ed'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:46.642808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-09e4ca85-4ff1-451d-8750-3ddf03d6d4ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9bcc2607bca49adb438229d27b252c8', 'request_id': 'req-09e4ca85-4ff1-451d-8750-3ddf03d6d4ed'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:46.712478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-09e4ca85-4ff1-451d-8750-3ddf03d6d4ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9bcc2607bca49adb438229d27b252c8', 'request_id': 'req-09e4ca85-4ff1-451d-8750-3ddf03d6d4ed'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:46.723817 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-09e4ca85-4ff1-451d-8750-3ddf03d6d4ed admin admin] The token cache is being invalidate because project b9bcc2607bca49adb438229d27b252c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:46.724108 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-09e4ca85-4ff1-451d-8750-3ddf03d6d4ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:46.724779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-09e4ca85-4ff1-451d-8750-3ddf03d6d4ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:46.726324 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1353/2677] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:46 2022] DELETE /identity/v3/projects/b9bcc2607bca49adb438229d27b252c8 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:46.731423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff283593-9bcf-4ed4-be04-db904f515c00 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:46.731921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff283593-9bcf-4ed4-be04-db904f515c00 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:46.732019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff283593-9bcf-4ed4-be04-db904f515c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:46.749693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ff283593-9bcf-4ed4-be04-db904f515c00 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:46.750921 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ff283593-9bcf-4ed4-be04-db904f515c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:46.815142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1325/2678] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:46.823738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f89ce4-2c6a-4d57-99ba-0b932f519dbd None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:46.824113 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f89ce4-2c6a-4d57-99ba-0b932f519dbd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:46.824473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f89ce4-2c6a-4d57-99ba-0b932f519dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:46.835145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:46.845236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-11f89ce4-2c6a-4d57-99ba-0b932f519dbd None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:46.846447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-11f89ce4-2c6a-4d57-99ba-0b932f519dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:46.854645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1354/2679] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:46.858886 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff283593-9bcf-4ed4-be04-db904f515c00 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:46.869888 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff283593-9bcf-4ed4-be04-db904f515c00 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:46.874251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555334a8-5f17-415b-be93-1987d57f760e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:46.875007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555334a8-5f17-415b-be93-1987d57f760e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:46.875389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555334a8-5f17-415b-be93-1987d57f760e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:46.876006 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555334a8-5f17-415b-be93-1987d57f760e admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:46.876585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555334a8-5f17-415b-be93-1987d57f760e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:24:46.876891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555334a8-5f17-415b-be93-1987d57f760e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:46.878478 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555334a8-5f17-415b-be93-1987d57f760e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:46.891653 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1326/2680] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:24:46.895929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f89ce4-2c6a-4d57-99ba-0b932f519dbd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:46.906747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f89ce4-2c6a-4d57-99ba-0b932f519dbd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:46.912330 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b1101-e728-42b2-8cb3-91956fde83e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:46.913044 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b1101-e728-42b2-8cb3-91956fde83e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:46.913921 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b1101-e728-42b2-8cb3-91956fde83e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:46.913921 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b1101-e728-42b2-8cb3-91956fde83e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:46.914455 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b1101-e728-42b2-8cb3-91956fde83e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:46.917068 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b1101-e728-42b2-8cb3-91956fde83e5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:46.988444 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1355/2681] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:46 2022] POST /identity/v3/projects => generated 365 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:46.989587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555334a8-5f17-415b-be93-1987d57f760e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:46.999877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555334a8-5f17-415b-be93-1987d57f760e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:47.061244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad32968-c5ea-4682-94b4-67cf565fe35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:47.063527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad32968-c5ea-4682-94b4-67cf565fe35c admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:47.063527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad32968-c5ea-4682-94b4-67cf565fe35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:47.063527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad32968-c5ea-4682-94b4-67cf565fe35c admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:47.063527 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad32968-c5ea-4682-94b4-67cf565fe35c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:47.065334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad32968-c5ea-4682-94b4-67cf565fe35c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:47.147333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:47.153676 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1327/2682] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:46 2022] POST /identity/v3/users => generated 283 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:47.158613 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b1101-e728-42b2-8cb3-91956fde83e5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:47.172603 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b1101-e728-42b2-8cb3-91956fde83e5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:47.177368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada104e-3b6b-426c-b2c8-f6bd4a77fa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:47.178194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada104e-3b6b-426c-b2c8-f6bd4a77fa07 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:47.178547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada104e-3b6b-426c-b2c8-f6bd4a77fa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:47.178989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada104e-3b6b-426c-b2c8-f6bd4a77fa07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:47.179930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada104e-3b6b-426c-b2c8-f6bd4a77fa07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:47.181913 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada104e-3b6b-426c-b2c8-f6bd4a77fa07 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:47.192728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1356/2683] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:47.198112 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad32968-c5ea-4682-94b4-67cf565fe35c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:47.209926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad32968-c5ea-4682-94b4-67cf565fe35c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:47.216533 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702c91e-3a34-4c8c-98d4-4a3d318ae766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:47.219012 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702c91e-3a34-4c8c-98d4-4a3d318ae766 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:47.219698 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702c91e-3a34-4c8c-98d4-4a3d318ae766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:47.219933 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702c91e-3a34-4c8c-98d4-4a3d318ae766 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:47.220742 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702c91e-3a34-4c8c-98d4-4a3d318ae766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:47.223141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702c91e-3a34-4c8c-98d4-4a3d318ae766 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:47.238626 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1328/2684] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:47.242801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada104e-3b6b-426c-b2c8-f6bd4a77fa07 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:47.253906 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada104e-3b6b-426c-b2c8-f6bd4a77fa07 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:47.259633 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9aac14-d6cf-4b49-ba10-dfd72f0dd2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:47.260771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9aac14-d6cf-4b49-ba10-dfd72f0dd2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:47.261123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9aac14-d6cf-4b49-ba10-dfd72f0dd2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:47.261581 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9aac14-d6cf-4b49-ba10-dfd72f0dd2c4 admin admin] PATH_INFO: `/v3/projects/6298fba65e38462998384fe3cca23b59/users/9608a96966044db0a3615b2ea2767018/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:47.281068 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9aac14-d6cf-4b49-ba10-dfd72f0dd2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6298fba65e38462998384fe3cca23b59, user_id=9608a96966044db0a3615b2ea2767018, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:47.283809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9aac14-d6cf-4b49-ba10-dfd72f0dd2c4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:47.355379 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1357/2685] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:47 2022] PUT /identity/v3/projects/6298fba65e38462998384fe3cca23b59/users/9608a96966044db0a3615b2ea2767018/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:47.365994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa074ddc-11ad-460b-bb2a-7b3579902ce7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:47.366284 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa074ddc-11ad-460b-bb2a-7b3579902ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:47.366564 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa074ddc-11ad-460b-bb2a-7b3579902ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:47.393960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-aa074ddc-11ad-460b-bb2a-7b3579902ce7 None None] MFA Rules not processed for user `9608a96966044db0a3615b2ea2767018`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:47.395194 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-aa074ddc-11ad-460b-bb2a-7b3579902ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:47.484110 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1329/2686] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:24:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:24:47.495955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9aac14-d6cf-4b49-ba10-dfd72f0dd2c4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:47.497792 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fa9aac14-d6cf-4b49-ba10-dfd72f0dd2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:47.582510 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9aac14-d6cf-4b49-ba10-dfd72f0dd2c4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:47.587367 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137fdbd-1a5a-4b34-aacd-6df3faa53e6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:47.588032 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137fdbd-1a5a-4b34-aacd-6df3faa53e6f service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:47.588342 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137fdbd-1a5a-4b34-aacd-6df3faa53e6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:47.588621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137fdbd-1a5a-4b34-aacd-6df3faa53e6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:47.598909 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137fdbd-1a5a-4b34-aacd-6df3faa53e6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:47.604147 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137fdbd-1a5a-4b34-aacd-6df3faa53e6f service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:47.616112 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1358/2687] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:49.507587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa074ddc-11ad-460b-bb2a-7b3579902ce7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:49.509752 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-aa074ddc-11ad-460b-bb2a-7b3579902ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:49.592356 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa074ddc-11ad-460b-bb2a-7b3579902ce7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:49.598105 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4134434-83c8-4c10-8553-f3c180cd9126 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:49.598809 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4134434-83c8-4c10-8553-f3c180cd9126 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:49.599101 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4134434-83c8-4c10-8553-f3c180cd9126 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:49.599402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4134434-83c8-4c10-8553-f3c180cd9126 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:49.601175 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b4134434-83c8-4c10-8553-f3c180cd9126 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:49.618106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4134434-83c8-4c10-8553-f3c180cd9126 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:49.619884 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4134434-83c8-4c10-8553-f3c180cd9126 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:49.631992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1330/2688] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:53.910214 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137fdbd-1a5a-4b34-aacd-6df3faa53e6f service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:53.911672 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2137fdbd-1a5a-4b34-aacd-6df3faa53e6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:53.967435 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137fdbd-1a5a-4b34-aacd-6df3faa53e6f service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:53.971118 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d5de5-f481-4eeb-aef5-562d7030dc14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:53.971789 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d5de5-f481-4eeb-aef5-562d7030dc14 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:53.972070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d5de5-f481-4eeb-aef5-562d7030dc14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:53.972345 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d5de5-f481-4eeb-aef5-562d7030dc14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:53.974125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e16d5de5-f481-4eeb-aef5-562d7030dc14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:54.040935 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:54.044707 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d5de5-f481-4eeb-aef5-562d7030dc14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:54.047469 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d5de5-f481-4eeb-aef5-562d7030dc14 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:54.058352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1359/2689] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:55.634117 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be2321-4129-4b46-a9a9-090480a187a7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:55.634644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be2321-4129-4b46-a9a9-090480a187a7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:55.634962 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be2321-4129-4b46-a9a9-090480a187a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:55.655431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-37be2321-4129-4b46-a9a9-090480a187a7 None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:55.657027 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-37be2321-4129-4b46-a9a9-090480a187a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:55.728476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1331/2690] 10.222.0.67 () {58 vars in 967 bytes} [Fri Jun 24 15:24:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:55.771847 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d5de5-f481-4eeb-aef5-562d7030dc14 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:55.773381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e16d5de5-f481-4eeb-aef5-562d7030dc14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:55.844789 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d5de5-f481-4eeb-aef5-562d7030dc14 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:55.849863 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d698a66-530d-4a6f-80b6-28da140f70ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:55.850520 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d698a66-530d-4a6f-80b6-28da140f70ac service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:55.850801 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d698a66-530d-4a6f-80b6-28da140f70ac service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:55.851078 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d698a66-530d-4a6f-80b6-28da140f70ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:55.859569 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d698a66-530d-4a6f-80b6-28da140f70ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:55.861306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d698a66-530d-4a6f-80b6-28da140f70ac service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:55.871081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1360/2691] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:24:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:25:00.678189 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f67c23-3d44-4f89-98e6-e5f7aae8e6a4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:00.678706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f67c23-3d44-4f89-98e6-e5f7aae8e6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:00.678796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f67c23-3d44-4f89-98e6-e5f7aae8e6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:00.705850 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d3f67c23-3d44-4f89-98e6-e5f7aae8e6a4 None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:25:00.707339 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d3f67c23-3d44-4f89-98e6-e5f7aae8e6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:00.716894 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1332/2692] 10.222.0.67 () {58 vars in 1009 bytes} [Fri Jun 24 15:25:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:25:00.732940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d698a66-530d-4a6f-80b6-28da140f70ac service swift] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:00.750717 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d698a66-530d-4a6f-80b6-28da140f70ac service swift] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:00.756836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b481268-784a-4808-9d79-9bb64f74e462 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:00.757487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b481268-784a-4808-9d79-9bb64f74e462 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:00.758269 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b481268-784a-4808-9d79-9bb64f74e462 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:00.758545 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b481268-784a-4808-9d79-9bb64f74e462 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:00.790567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b481268-784a-4808-9d79-9bb64f74e462 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:00.792278 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b481268-784a-4808-9d79-9bb64f74e462 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:00.802822 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1361/2693] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:25:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:25:06.451030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f67c23-3d44-4f89-98e6-e5f7aae8e6a4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:06.452896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d3f67c23-3d44-4f89-98e6-e5f7aae8e6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:06.460587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:06.577609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f67c23-3d44-4f89-98e6-e5f7aae8e6a4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:06.582740 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b0c30-b79a-4bb8-95a7-bb5f2d89a271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:06.584244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b0c30-b79a-4bb8-95a7-bb5f2d89a271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:06.584701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b0c30-b79a-4bb8-95a7-bb5f2d89a271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:06.585130 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b0c30-b79a-4bb8-95a7-bb5f2d89a271 admin admin] PATH_INFO: `/v3/users/e4f136ae627b4a81b6b29c667420f1aa` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:06.587197 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b0c30-b79a-4bb8-95a7-bb5f2d89a271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4f136ae627b4a81b6b29c667420f1aa)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:06.589109 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b0c30-b79a-4bb8-95a7-bb5f2d89a271 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:06.795780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-787b0c30-b79a-4bb8-95a7-bb5f2d89a271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4f136ae627b4a81b6b29c667420f1aa'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:06.812887 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-787b0c30-b79a-4bb8-95a7-bb5f2d89a271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4f136ae627b4a81b6b29c667420f1aa'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:06.813661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-787b0c30-b79a-4bb8-95a7-bb5f2d89a271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4f136ae627b4a81b6b29c667420f1aa'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:06.823198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-787b0c30-b79a-4bb8-95a7-bb5f2d89a271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4f136ae627b4a81b6b29c667420f1aa'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:06.823983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-787b0c30-b79a-4bb8-95a7-bb5f2d89a271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4f136ae627b4a81b6b29c667420f1aa'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:06.915797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1333/2694] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:25:06 2022] DELETE /identity/v3/users/e4f136ae627b4a81b6b29c667420f1aa => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:25:07.307163 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b481268-784a-4808-9d79-9bb64f74e462 service swift] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:07.318982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b481268-784a-4808-9d79-9bb64f74e462 service swift] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:07.416966 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb7bef-669c-4733-90f8-95d82aa8435f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:07.418106 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb7bef-669c-4733-90f8-95d82aa8435f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:07.418591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb7bef-669c-4733-90f8-95d82aa8435f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:07.419040 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb7bef-669c-4733-90f8-95d82aa8435f admin admin] PATH_INFO: `/v3/projects/fb2390854b0345588bd6b68f5c09a1ee` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:07.420582 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb7bef-669c-4733-90f8-95d82aa8435f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb2390854b0345588bd6b68f5c09a1ee)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:07.422845 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb7bef-669c-4733-90f8-95d82aa8435f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:07.567020 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6ecb7bef-669c-4733-90f8-95d82aa8435f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb2390854b0345588bd6b68f5c09a1ee', 'request_id': 'req-6ecb7bef-669c-4733-90f8-95d82aa8435f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:07.575467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6ecb7bef-669c-4733-90f8-95d82aa8435f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb2390854b0345588bd6b68f5c09a1ee', 'request_id': 'req-6ecb7bef-669c-4733-90f8-95d82aa8435f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:07.649975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6ecb7bef-669c-4733-90f8-95d82aa8435f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb2390854b0345588bd6b68f5c09a1ee', 'request_id': 'req-6ecb7bef-669c-4733-90f8-95d82aa8435f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:07.666526 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:07.670939 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6ecb7bef-669c-4733-90f8-95d82aa8435f admin admin] The token cache is being invalidate because project fb2390854b0345588bd6b68f5c09a1ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:25:07.671347 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6ecb7bef-669c-4733-90f8-95d82aa8435f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:07.672087 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6ecb7bef-669c-4733-90f8-95d82aa8435f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:07.673791 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1362/2695] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:25:07 2022] DELETE /identity/v3/projects/fb2390854b0345588bd6b68f5c09a1ee => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:25:07.677930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b0c30-b79a-4bb8-95a7-bb5f2d89a271 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:07.679416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-787b0c30-b79a-4bb8-95a7-bb5f2d89a271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:07.694054 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b0c30-b79a-4bb8-95a7-bb5f2d89a271 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:07.783829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5daf2a-e803-43ca-9b4e-4cbff63c8ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:07.784835 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5daf2a-e803-43ca-9b4e-4cbff63c8ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:07.785489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5daf2a-e803-43ca-9b4e-4cbff63c8ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:07.785489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5daf2a-e803-43ca-9b4e-4cbff63c8ec2 admin admin] PATH_INFO: `/v3/users/40fb07886b4c4054a129d34670ef9742` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:07.786897 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5daf2a-e803-43ca-9b4e-4cbff63c8ec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40fb07886b4c4054a129d34670ef9742)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:07.790686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5daf2a-e803-43ca-9b4e-4cbff63c8ec2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:07.839517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:08.348722 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6c5daf2a-e803-43ca-9b4e-4cbff63c8ec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40fb07886b4c4054a129d34670ef9742'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:08.369472 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6c5daf2a-e803-43ca-9b4e-4cbff63c8ec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40fb07886b4c4054a129d34670ef9742'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:08.373097 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6c5daf2a-e803-43ca-9b4e-4cbff63c8ec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40fb07886b4c4054a129d34670ef9742'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:08.377399 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6c5daf2a-e803-43ca-9b4e-4cbff63c8ec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40fb07886b4c4054a129d34670ef9742'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:08.378054 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6c5daf2a-e803-43ca-9b4e-4cbff63c8ec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40fb07886b4c4054a129d34670ef9742'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:08.443426 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1334/2696] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:25:07 2022] DELETE /identity/v3/users/40fb07886b4c4054a129d34670ef9742 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:25:08.821570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb7bef-669c-4733-90f8-95d82aa8435f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:08.821570 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6ecb7bef-669c-4733-90f8-95d82aa8435f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:08.845875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb7bef-669c-4733-90f8-95d82aa8435f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:08.953934 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e9090-1c51-418c-aa17-8c93edb40e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:08.954843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e9090-1c51-418c-aa17-8c93edb40e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:08.955130 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e9090-1c51-418c-aa17-8c93edb40e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:08.955445 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e9090-1c51-418c-aa17-8c93edb40e9d admin admin] PATH_INFO: `/v3/projects/d5ed08c77e8247d0abc0ffefda355d6b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:08.956949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e9090-1c51-418c-aa17-8c93edb40e9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5ed08c77e8247d0abc0ffefda355d6b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:08.958907 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e9090-1c51-418c-aa17-8c93edb40e9d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:09.089834 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a08e9090-1c51-418c-aa17-8c93edb40e9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5ed08c77e8247d0abc0ffefda355d6b', 'request_id': 'req-a08e9090-1c51-418c-aa17-8c93edb40e9d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:09.097173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a08e9090-1c51-418c-aa17-8c93edb40e9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5ed08c77e8247d0abc0ffefda355d6b', 'request_id': 'req-a08e9090-1c51-418c-aa17-8c93edb40e9d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:09.160141 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a08e9090-1c51-418c-aa17-8c93edb40e9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5ed08c77e8247d0abc0ffefda355d6b', 'request_id': 'req-a08e9090-1c51-418c-aa17-8c93edb40e9d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:09.174358 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a08e9090-1c51-418c-aa17-8c93edb40e9d admin admin] The token cache is being invalidate because project d5ed08c77e8247d0abc0ffefda355d6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:25:09.174780 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a08e9090-1c51-418c-aa17-8c93edb40e9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:09.175634 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a08e9090-1c51-418c-aa17-8c93edb40e9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:09.177222 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1363/2697] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:25:08 2022] DELETE /identity/v3/projects/d5ed08c77e8247d0abc0ffefda355d6b => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:25:14.299484 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5daf2a-e803-43ca-9b4e-4cbff63c8ec2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:14.300910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6c5daf2a-e803-43ca-9b4e-4cbff63c8ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:14.364914 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5daf2a-e803-43ca-9b4e-4cbff63c8ec2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:14.423000 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2923a-5bad-485b-ac9a-82c1fc206b19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:14.423695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2923a-5bad-485b-ac9a-82c1fc206b19 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:14.423982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2923a-5bad-485b-ac9a-82c1fc206b19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:14.424263 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2923a-5bad-485b-ac9a-82c1fc206b19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:14.426154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-79c2923a-5bad-485b-ac9a-82c1fc206b19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:14.437413 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2923a-5bad-485b-ac9a-82c1fc206b19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:14.439018 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2923a-5bad-485b-ac9a-82c1fc206b19 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:14.449552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1335/2698] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:25:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:25:19.170430 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:19.848021 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78323eb3-18ee-4342-ad0f-3734d9c8380c None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:19.848307 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78323eb3-18ee-4342-ad0f-3734d9c8380c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:19.848609 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78323eb3-18ee-4342-ad0f-3734d9c8380c None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:19.850029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1364/2699] 10.222.0.67 () {54 vars in 921 bytes} [Fri Jun 24 15:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:25:19.857231 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e118a4-afeb-40bd-9ba6-4f85d4e0f1f7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:19.857781 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e118a4-afeb-40bd-9ba6-4f85d4e0f1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:19.857781 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e118a4-afeb-40bd-9ba6-4f85d4e0f1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:19.876729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-10e118a4-afeb-40bd-9ba6-4f85d4e0f1f7 None None] MFA Rules not processed for user `51de59caa3144a1cbf12b20b15e2134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:25:19.879632 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-10e118a4-afeb-40bd-9ba6-4f85d4e0f1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:19.932606 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:19.956567 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1336/2700] 10.222.0.67 () {58 vars in 1018 bytes} [Fri Jun 24 15:25:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:25:19.968680 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78323eb3-18ee-4342-ad0f-3734d9c8380c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:19.971317 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-78323eb3-18ee-4342-ad0f-3734d9c8380c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:20.034274 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78323eb3-18ee-4342-ad0f-3734d9c8380c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:20.034673 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087df93-39f6-470f-acb1-02e0a626b0e6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1971c464293f426f97d6cb199e48193b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:20.035397 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087df93-39f6-470f-acb1-02e0a626b0e6 service glance] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:20.035685 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087df93-39f6-470f-acb1-02e0a626b0e6 service glance] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:20.035968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087df93-39f6-470f-acb1-02e0a626b0e6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:20.046203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087df93-39f6-470f-acb1-02e0a626b0e6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:20.047647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087df93-39f6-470f-acb1-02e0a626b0e6 service glance] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:20.059341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1365/2701] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:25:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:25:20.250714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563db3b0-a600-4fa2-992f-d06736b049f2 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:20.251005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563db3b0-a600-4fa2-992f-d06736b049f2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:20.251323 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563db3b0-a600-4fa2-992f-d06736b049f2 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:20.252712 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1337/2702] 10.222.0.67 () {54 vars in 921 bytes} [Fri Jun 24 15:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:25:20.257571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7f85d-d394-470b-9456-45cb2d89edfb None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:20.258073 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7f85d-d394-470b-9456-45cb2d89edfb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:20.258186 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7f85d-d394-470b-9456-45cb2d89edfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:20.277042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6bf7f85d-d394-470b-9456-45cb2d89edfb None None] MFA Rules not processed for user `95e7fb8f48dd4348b54e3694a8410bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:25:20.278264 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6bf7f85d-d394-470b-9456-45cb2d89edfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:20.284914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1366/2703] 10.222.0.67 () {58 vars in 1018 bytes} [Fri Jun 24 15:25:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:25:20.297073 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563db3b0-a600-4fa2-992f-d06736b049f2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:20.299155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-563db3b0-a600-4fa2-992f-d06736b049f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:20.313233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563db3b0-a600-4fa2-992f-d06736b049f2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:20.316746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1aea4c-ac52-4307-a227-ae570deebe52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:20.317385 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1aea4c-ac52-4307-a227-ae570deebe52 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:20.317673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1aea4c-ac52-4307-a227-ae570deebe52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:20.317950 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1aea4c-ac52-4307-a227-ae570deebe52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:20.325254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1aea4c-ac52-4307-a227-ae570deebe52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:20.326686 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1aea4c-ac52-4307-a227-ae570deebe52 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:20.334407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1338/2704] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:25:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:25:20.416702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f6ff6-f0dd-4873-a584-5683c8b1a9da None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:20.416994 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f6ff6-f0dd-4873-a584-5683c8b1a9da None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:20.417277 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f6ff6-f0dd-4873-a584-5683c8b1a9da None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:20.418648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1367/2705] 10.222.0.67 () {54 vars in 921 bytes} [Fri Jun 24 15:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:25:20.424344 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d519399-3a09-48a9-af03-36e178e64d9f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:20.425188 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d519399-3a09-48a9-af03-36e178e64d9f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:20.425188 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d519399-3a09-48a9-af03-36e178e64d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:20.444788 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1d519399-3a09-48a9-af03-36e178e64d9f None None] MFA Rules not processed for user `51de59caa3144a1cbf12b20b15e2134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:25:20.445989 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1d519399-3a09-48a9-af03-36e178e64d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:20.455591 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1339/2706] 10.222.0.67 () {58 vars in 1018 bytes} [Fri Jun 24 15:25:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:25:20.466380 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f6ff6-f0dd-4873-a584-5683c8b1a9da None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:20.478032 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f6ff6-f0dd-4873-a584-5683c8b1a9da None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:20.481751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d14b0-c3cd-4786-a5ba-1d5ea9442298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:20.482384 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d14b0-c3cd-4786-a5ba-1d5ea9442298 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:20.484063 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d14b0-c3cd-4786-a5ba-1d5ea9442298 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:20.484063 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d14b0-c3cd-4786-a5ba-1d5ea9442298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:20.494460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d14b0-c3cd-4786-a5ba-1d5ea9442298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:20.495966 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d14b0-c3cd-4786-a5ba-1d5ea9442298 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:20.506783 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1368/2707] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:25:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:25:22.652234 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ffc7f2-82ce-4481-b6cf-a32d37bef24c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:22.652717 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ffc7f2-82ce-4481-b6cf-a32d37bef24c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:22.652827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ffc7f2-82ce-4481-b6cf-a32d37bef24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:22.671069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-08ffc7f2-82ce-4481-b6cf-a32d37bef24c None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:25:22.672295 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-08ffc7f2-82ce-4481-b6cf-a32d37bef24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:22.729471 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1340/2708] 10.222.0.67 () {58 vars in 1009 bytes} [Fri Jun 24 15:25:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:25:22.742345 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d14b0-c3cd-4786-a5ba-1d5ea9442298 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:22.743837 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8e2d14b0-c3cd-4786-a5ba-1d5ea9442298 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:22.814542 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d14b0-c3cd-4786-a5ba-1d5ea9442298 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:22.819651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d596fd-83b7-460b-8f32-e17e63f581d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:22.820492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d596fd-83b7-460b-8f32-e17e63f581d8 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:22.820776 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d596fd-83b7-460b-8f32-e17e63f581d8 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:22.821060 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d596fd-83b7-460b-8f32-e17e63f581d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:22.829668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d596fd-83b7-460b-8f32-e17e63f581d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:22.835998 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d596fd-83b7-460b-8f32-e17e63f581d8 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:22.840627 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1369/2709] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:25:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:25:28.648432 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ffc7f2-82ce-4481-b6cf-a32d37bef24c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:28.650329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-08ffc7f2-82ce-4481-b6cf-a32d37bef24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:28.721703 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ffc7f2-82ce-4481-b6cf-a32d37bef24c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:28.726751 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83480d-f199-4ee0-8747-3b0810c741ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:28.727865 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83480d-f199-4ee0-8747-3b0810c741ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:28.728154 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83480d-f199-4ee0-8747-3b0810c741ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:28.728422 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83480d-f199-4ee0-8747-3b0810c741ae admin admin] PATH_INFO: `/v3/users/9608a96966044db0a3615b2ea2767018` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:28.732165 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83480d-f199-4ee0-8747-3b0810c741ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=9608a96966044db0a3615b2ea2767018)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:28.732165 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83480d-f199-4ee0-8747-3b0810c741ae admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:28.889649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9e83480d-f199-4ee0-8747-3b0810c741ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9608a96966044db0a3615b2ea2767018'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:28.904414 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9e83480d-f199-4ee0-8747-3b0810c741ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9608a96966044db0a3615b2ea2767018'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:28.905141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9e83480d-f199-4ee0-8747-3b0810c741ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9608a96966044db0a3615b2ea2767018'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:28.913040 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9e83480d-f199-4ee0-8747-3b0810c741ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9608a96966044db0a3615b2ea2767018'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:28.914528 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9e83480d-f199-4ee0-8747-3b0810c741ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9608a96966044db0a3615b2ea2767018'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:28.919335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:29.001068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1341/2710] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:25:28 2022] DELETE /identity/v3/users/9608a96966044db0a3615b2ea2767018 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:25:29.002828 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beba053-e149-449d-9304-57814c8d723c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:29.003549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beba053-e149-449d-9304-57814c8d723c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:29.003549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beba053-e149-449d-9304-57814c8d723c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:29.026004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3beba053-e149-449d-9304-57814c8d723c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:25:29.027308 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3beba053-e149-449d-9304-57814c8d723c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:29.122174 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1370/2711] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:25:29.134034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83480d-f199-4ee0-8747-3b0810c741ae admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:29.136331 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9e83480d-f199-4ee0-8747-3b0810c741ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:29.211024 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83480d-f199-4ee0-8747-3b0810c741ae admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:29.214956 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd829f-568a-4e92-b333-1e0db9b3a036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:29.215648 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd829f-568a-4e92-b333-1e0db9b3a036 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:29.215928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd829f-568a-4e92-b333-1e0db9b3a036 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:29.216184 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd829f-568a-4e92-b333-1e0db9b3a036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:29.227683 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd829f-568a-4e92-b333-1e0db9b3a036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:29.229132 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd829f-568a-4e92-b333-1e0db9b3a036 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:29.239060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1342/2712] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:25:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:25:29.703257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beba053-e149-449d-9304-57814c8d723c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:29.704805 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3beba053-e149-449d-9304-57814c8d723c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:29.726876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beba053-e149-449d-9304-57814c8d723c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:29.739419 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769657a-b30b-42cb-8559-6cb34acc229c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:29.740471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769657a-b30b-42cb-8559-6cb34acc229c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:29.740835 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769657a-b30b-42cb-8559-6cb34acc229c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:29.741967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769657a-b30b-42cb-8559-6cb34acc229c admin admin] PATH_INFO: `/v3/projects/6298fba65e38462998384fe3cca23b59` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:29.743250 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769657a-b30b-42cb-8559-6cb34acc229c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6298fba65e38462998384fe3cca23b59)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:29.745238 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769657a-b30b-42cb-8559-6cb34acc229c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:29.754379 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:29.853370 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5769657a-b30b-42cb-8559-6cb34acc229c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6298fba65e38462998384fe3cca23b59', 'request_id': 'req-5769657a-b30b-42cb-8559-6cb34acc229c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:29.860309 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5769657a-b30b-42cb-8559-6cb34acc229c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6298fba65e38462998384fe3cca23b59', 'request_id': 'req-5769657a-b30b-42cb-8559-6cb34acc229c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:29.934664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5769657a-b30b-42cb-8559-6cb34acc229c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6298fba65e38462998384fe3cca23b59', 'request_id': 'req-5769657a-b30b-42cb-8559-6cb34acc229c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:29.946544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5769657a-b30b-42cb-8559-6cb34acc229c admin admin] The token cache is being invalidate because project 6298fba65e38462998384fe3cca23b59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:25:29.946753 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5769657a-b30b-42cb-8559-6cb34acc229c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:29.947432 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5769657a-b30b-42cb-8559-6cb34acc229c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:29.949381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1371/2713] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:25:29 2022] DELETE /identity/v3/projects/6298fba65e38462998384fe3cca23b59 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:25:29.954118 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328dd36-9139-404a-848c-65dfccb7f634 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:29.954669 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328dd36-9139-404a-848c-65dfccb7f634 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:29.954777 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328dd36-9139-404a-848c-65dfccb7f634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:29.974343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9328dd36-9139-404a-848c-65dfccb7f634 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:25:29.976424 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9328dd36-9139-404a-848c-65dfccb7f634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:30.047285 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1343/2714] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:25:30.050210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4a665-6a66-4543-9243-d2fd49d8af01 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:30.050811 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4a665-6a66-4543-9243-d2fd49d8af01 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:30.050811 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4a665-6a66-4543-9243-d2fd49d8af01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:30.071524 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c1e4a665-6a66-4543-9243-d2fd49d8af01 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:25:30.072731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c1e4a665-6a66-4543-9243-d2fd49d8af01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:30.081042 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1372/2715] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:25:30.086014 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328dd36-9139-404a-848c-65dfccb7f634 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:30.102513 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328dd36-9139-404a-848c-65dfccb7f634 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:30.108234 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b8a2b-2c4c-40d1-b441-3a35f60b809a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:30.109061 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b8a2b-2c4c-40d1-b441-3a35f60b809a admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:30.109586 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b8a2b-2c4c-40d1-b441-3a35f60b809a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:30.109916 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b8a2b-2c4c-40d1-b441-3a35f60b809a admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:30.110725 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b8a2b-2c4c-40d1-b441-3a35f60b809a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:25:30.111060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b8a2b-2c4c-40d1-b441-3a35f60b809a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:30.112729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b8a2b-2c4c-40d1-b441-3a35f60b809a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:30.126445 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1344/2716] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:25:30.130653 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4a665-6a66-4543-9243-d2fd49d8af01 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:30.141616 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4a665-6a66-4543-9243-d2fd49d8af01 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:30.158397 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4802ff4-f711-410d-ba36-891216166c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:30.158892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4802ff4-f711-410d-ba36-891216166c3e admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:30.162105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4802ff4-f711-410d-ba36-891216166c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:30.162105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4802ff4-f711-410d-ba36-891216166c3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:30.162105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4802ff4-f711-410d-ba36-891216166c3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:30.164485 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4802ff4-f711-410d-ba36-891216166c3e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:30.263006 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1373/2717] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:25:30 2022] POST /identity/v3/projects => generated 399 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:25:30.268304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b8a2b-2c4c-40d1-b441-3a35f60b809a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:30.279425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b8a2b-2c4c-40d1-b441-3a35f60b809a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:30.391797 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa1220-aaff-46a1-b215-63832d01cf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:30.392789 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa1220-aaff-46a1-b215-63832d01cf80 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:30.393173 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa1220-aaff-46a1-b215-63832d01cf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:30.393851 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa1220-aaff-46a1-b215-63832d01cf80 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:30.396034 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa1220-aaff-46a1-b215-63832d01cf80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:30.398903 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa1220-aaff-46a1-b215-63832d01cf80 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:30.501335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1345/2718] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:25:30 2022] POST /identity/v3/users => generated 300 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:25:30.502892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4802ff4-f711-410d-ba36-891216166c3e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:30.528300 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4802ff4-f711-410d-ba36-891216166c3e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:30.528300 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ceef96-11aa-4678-9127-88a84b042303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:30.528300 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ceef96-11aa-4678-9127-88a84b042303 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:30.528300 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ceef96-11aa-4678-9127-88a84b042303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:30.528873 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ceef96-11aa-4678-9127-88a84b042303 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:30.529280 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ceef96-11aa-4678-9127-88a84b042303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:30.530875 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ceef96-11aa-4678-9127-88a84b042303 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:30.546843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1374/2719] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:25:30.548533 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa1220-aaff-46a1-b215-63832d01cf80 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:30.565074 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa1220-aaff-46a1-b215-63832d01cf80 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:30.571638 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a9660-9d9b-47e1-a206-70bc74d11d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:30.572711 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a9660-9d9b-47e1-a206-70bc74d11d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:30.573069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a9660-9d9b-47e1-a206-70bc74d11d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:30.573354 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a9660-9d9b-47e1-a206-70bc74d11d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:30.574341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a9660-9d9b-47e1-a206-70bc74d11d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:30.576009 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a9660-9d9b-47e1-a206-70bc74d11d59 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:30.595058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1346/2720] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:25:30.596700 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ceef96-11aa-4678-9127-88a84b042303 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:30.608598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ceef96-11aa-4678-9127-88a84b042303 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:30.616747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf0c85-3b1b-45d7-b0df-347874c5a358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:30.620018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf0c85-3b1b-45d7-b0df-347874c5a358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:30.620018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf0c85-3b1b-45d7-b0df-347874c5a358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:30.620018 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf0c85-3b1b-45d7-b0df-347874c5a358 admin admin] PATH_INFO: `/v3/projects/58dafacaaed04bcdbd07dcef57dc5e71/users/55add621f44d41b4b707f94d6c0fe389/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:30.640171 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf0c85-3b1b-45d7-b0df-347874c5a358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58dafacaaed04bcdbd07dcef57dc5e71, user_id=55add621f44d41b4b707f94d6c0fe389, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:30.643200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf0c85-3b1b-45d7-b0df-347874c5a358 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:30.721006 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1375/2721] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:25:30 2022] PUT /identity/v3/projects/58dafacaaed04bcdbd07dcef57dc5e71/users/55add621f44d41b4b707f94d6c0fe389/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:25:30.732389 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365805d5-f080-4f05-8951-c618255db857 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:30.737085 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365805d5-f080-4f05-8951-c618255db857 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:30.737085 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365805d5-f080-4f05-8951-c618255db857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:30.742070 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:30.764796 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-365805d5-f080-4f05-8951-c618255db857 None None] MFA Rules not processed for user `55add621f44d41b4b707f94d6c0fe389`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:25:30.767379 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-365805d5-f080-4f05-8951-c618255db857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:30.847224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1347/2722] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:25:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:25:30.860551 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf0c85-3b1b-45d7-b0df-347874c5a358 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:30.862979 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-86cf0c85-3b1b-45d7-b0df-347874c5a358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:30.930678 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf0c85-3b1b-45d7-b0df-347874c5a358 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:30.935869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69792ccd-f653-4811-9c15-b6609a00403d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:30.936702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69792ccd-f653-4811-9c15-b6609a00403d service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:30.937035 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69792ccd-f653-4811-9c15-b6609a00403d service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:30.937347 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69792ccd-f653-4811-9c15-b6609a00403d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:30.948221 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69792ccd-f653-4811-9c15-b6609a00403d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:30.950184 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69792ccd-f653-4811-9c15-b6609a00403d service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:30.961988 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1376/2723] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:25:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:25:39.765166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365805d5-f080-4f05-8951-c618255db857 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:39.828216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365805d5-f080-4f05-8951-c618255db857 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:39.832794 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956508d-e2f6-4f95-99d0-7fc838e9283c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:39.833658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956508d-e2f6-4f95-99d0-7fc838e9283c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:39.833939 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956508d-e2f6-4f95-99d0-7fc838e9283c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:39.834213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956508d-e2f6-4f95-99d0-7fc838e9283c admin admin] PATH_INFO: `/v3/users/55add621f44d41b4b707f94d6c0fe389` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:39.835395 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956508d-e2f6-4f95-99d0-7fc838e9283c admin admin] RBAC: Authorizing `identity:delete_user(user_id=55add621f44d41b4b707f94d6c0fe389)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:39.837470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956508d-e2f6-4f95-99d0-7fc838e9283c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:40.056248 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f956508d-e2f6-4f95-99d0-7fc838e9283c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55add621f44d41b4b707f94d6c0fe389'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:40.070250 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f956508d-e2f6-4f95-99d0-7fc838e9283c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55add621f44d41b4b707f94d6c0fe389'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:40.070988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f956508d-e2f6-4f95-99d0-7fc838e9283c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55add621f44d41b4b707f94d6c0fe389'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:40.078497 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f956508d-e2f6-4f95-99d0-7fc838e9283c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55add621f44d41b4b707f94d6c0fe389'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:40.079142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f956508d-e2f6-4f95-99d0-7fc838e9283c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55add621f44d41b4b707f94d6c0fe389'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:40.139770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1348/2724] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:25:39 2022] DELETE /identity/v3/users/55add621f44d41b4b707f94d6c0fe389 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:25:40.144305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b341d5b-9d67-4044-bab5-3cae0fc6d6ca None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:40.144796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b341d5b-9d67-4044-bab5-3cae0fc6d6ca None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:40.144893 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b341d5b-9d67-4044-bab5-3cae0fc6d6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:40.163486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8b341d5b-9d67-4044-bab5-3cae0fc6d6ca None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:25:40.165054 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8b341d5b-9d67-4044-bab5-3cae0fc6d6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:40.185682 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:40.223913 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1377/2725] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:25:40.234398 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956508d-e2f6-4f95-99d0-7fc838e9283c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:40.235831 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f956508d-e2f6-4f95-99d0-7fc838e9283c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:40.250371 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:40.293479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956508d-e2f6-4f95-99d0-7fc838e9283c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:40.297203 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14164e4-b885-49b0-b3a6-6a3615a5967b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:40.297859 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14164e4-b885-49b0-b3a6-6a3615a5967b service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:40.298141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14164e4-b885-49b0-b3a6-6a3615a5967b service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:40.298420 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14164e4-b885-49b0-b3a6-6a3615a5967b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:40.307814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14164e4-b885-49b0-b3a6-6a3615a5967b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:40.309254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14164e4-b885-49b0-b3a6-6a3615a5967b service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:40.319768 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1349/2726] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:25:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:25:40.483522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba0f9e-eb7b-4a1d-a16a-815ffa6d0228 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:40.483844 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba0f9e-eb7b-4a1d-a16a-815ffa6d0228 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:40.484167 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba0f9e-eb7b-4a1d-a16a-815ffa6d0228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:40.504116 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7bba0f9e-eb7b-4a1d-a16a-815ffa6d0228 None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:25:40.505334 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7bba0f9e-eb7b-4a1d-a16a-815ffa6d0228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:40.565982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1378/2727] 10.222.0.67 () {58 vars in 967 bytes} [Fri Jun 24 15:25:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:25:40.579615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14164e4-b885-49b0-b3a6-6a3615a5967b service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:40.581094 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a14164e4-b885-49b0-b3a6-6a3615a5967b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:40.658718 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14164e4-b885-49b0-b3a6-6a3615a5967b service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:40.663756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e7edd-7586-49da-9656-56b9b826cd62 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:40.664431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e7edd-7586-49da-9656-56b9b826cd62 service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:40.664712 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e7edd-7586-49da-9656-56b9b826cd62 service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:40.665054 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e7edd-7586-49da-9656-56b9b826cd62 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:40.674918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e7edd-7586-49da-9656-56b9b826cd62 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:40.676593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e7edd-7586-49da-9656-56b9b826cd62 service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:40.686108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1350/2728] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:25:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:25:40.969474 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba0f9e-eb7b-4a1d-a16a-815ffa6d0228 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:40.970946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7bba0f9e-eb7b-4a1d-a16a-815ffa6d0228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:40.985921 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba0f9e-eb7b-4a1d-a16a-815ffa6d0228 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:40.991080 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf006837-6d45-42ee-b18c-2154a0acd679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:40.991983 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf006837-6d45-42ee-b18c-2154a0acd679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:40.992263 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf006837-6d45-42ee-b18c-2154a0acd679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:40.992541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf006837-6d45-42ee-b18c-2154a0acd679 admin admin] PATH_INFO: `/v3/projects/58dafacaaed04bcdbd07dcef57dc5e71` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:40.994218 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf006837-6d45-42ee-b18c-2154a0acd679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58dafacaaed04bcdbd07dcef57dc5e71)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:40.996165 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf006837-6d45-42ee-b18c-2154a0acd679 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:41.110152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cf006837-6d45-42ee-b18c-2154a0acd679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58dafacaaed04bcdbd07dcef57dc5e71', 'request_id': 'req-cf006837-6d45-42ee-b18c-2154a0acd679'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:41.117721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cf006837-6d45-42ee-b18c-2154a0acd679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58dafacaaed04bcdbd07dcef57dc5e71', 'request_id': 'req-cf006837-6d45-42ee-b18c-2154a0acd679'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:41.197136 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cf006837-6d45-42ee-b18c-2154a0acd679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58dafacaaed04bcdbd07dcef57dc5e71', 'request_id': 'req-cf006837-6d45-42ee-b18c-2154a0acd679'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:41.210110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cf006837-6d45-42ee-b18c-2154a0acd679 admin admin] The token cache is being invalidate because project 58dafacaaed04bcdbd07dcef57dc5e71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:25:41.210110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cf006837-6d45-42ee-b18c-2154a0acd679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:41.210714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cf006837-6d45-42ee-b18c-2154a0acd679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:41.212521 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1379/2729] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:25:40 2022] DELETE /identity/v3/projects/58dafacaaed04bcdbd07dcef57dc5e71 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:25:50.682076 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=52 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:51.206185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=67,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:26:22.962616 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e7edd-7586-49da-9656-56b9b826cd62 service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:26:22.964212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-698e7edd-7586-49da-9656-56b9b826cd62 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:26:22.976141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e7edd-7586-49da-9656-56b9b826cd62 service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:26:23.025213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7af4c-bb74-4337-a1d1-ad8e78fb26c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:26:23.026415 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7af4c-bb74-4337-a1d1-ad8e78fb26c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:23.026692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7af4c-bb74-4337-a1d1-ad8e78fb26c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:23.026968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7af4c-bb74-4337-a1d1-ad8e78fb26c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:23.028875 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0bd7af4c-bb74-4337-a1d1-ad8e78fb26c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:26:23.091531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7af4c-bb74-4337-a1d1-ad8e78fb26c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:26:23.092987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7af4c-bb74-4337-a1d1-ad8e78fb26c0 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:26:23.103076 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1351/2730] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:26:26.543454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf006837-6d45-42ee-b18c-2154a0acd679 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:26:26.544919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cf006837-6d45-42ee-b18c-2154a0acd679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:26:26.615759 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf006837-6d45-42ee-b18c-2154a0acd679 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:26:26.621011 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e0f5f-b9ad-4f95-808a-04f9b9407556 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:26:26.621669 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e0f5f-b9ad-4f95-808a-04f9b9407556 service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:26.621956 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e0f5f-b9ad-4f95-808a-04f9b9407556 service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:26.622236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e0f5f-b9ad-4f95-808a-04f9b9407556 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:26.624149 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-248e0f5f-b9ad-4f95-808a-04f9b9407556 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:26:26.635777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e0f5f-b9ad-4f95-808a-04f9b9407556 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:26:26.639207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e0f5f-b9ad-4f95-808a-04f9b9407556 service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:26:26.647460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1380/2731] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:26:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:26:26.693291 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7af4c-bb74-4337-a1d1-ad8e78fb26c0 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:26:26.694744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0bd7af4c-bb74-4337-a1d1-ad8e78fb26c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:26:26.708640 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7af4c-bb74-4337-a1d1-ad8e78fb26c0 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:26:26.713890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ea216-90cc-45cf-9554-985e8d9d78c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:26:26.714738 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ea216-90cc-45cf-9554-985e8d9d78c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:26.715022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ea216-90cc-45cf-9554-985e8d9d78c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:26.715333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ea216-90cc-45cf-9554-985e8d9d78c7 admin admin] PATH_INFO: `/v3/users/206ca9f278fc47e4a9a4f720083e91f3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:26.716514 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ea216-90cc-45cf-9554-985e8d9d78c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=206ca9f278fc47e4a9a4f720083e91f3)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:26:26.718442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ea216-90cc-45cf-9554-985e8d9d78c7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:26:26.901140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-047ea216-90cc-45cf-9554-985e8d9d78c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '206ca9f278fc47e4a9a4f720083e91f3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:26:26.913446 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-047ea216-90cc-45cf-9554-985e8d9d78c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '206ca9f278fc47e4a9a4f720083e91f3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:26:26.914168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-047ea216-90cc-45cf-9554-985e8d9d78c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '206ca9f278fc47e4a9a4f720083e91f3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:26:26.921135 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-047ea216-90cc-45cf-9554-985e8d9d78c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '206ca9f278fc47e4a9a4f720083e91f3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:26:26.921776 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-047ea216-90cc-45cf-9554-985e8d9d78c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '206ca9f278fc47e4a9a4f720083e91f3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:26:26.982612 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1352/2732] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:26:26 2022] DELETE /identity/v3/users/206ca9f278fc47e4a9a4f720083e91f3 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:26:27.257779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e0f5f-b9ad-4f95-808a-04f9b9407556 service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:26:27.269494 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e0f5f-b9ad-4f95-808a-04f9b9407556 service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:26:27.330118 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103682f-558e-4722-bff5-e1c06160f1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:26:27.330981 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103682f-558e-4722-bff5-e1c06160f1fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:27.331294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103682f-558e-4722-bff5-e1c06160f1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:27.331585 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103682f-558e-4722-bff5-e1c06160f1fb admin admin] PATH_INFO: `/v3/projects/c4807ce933f14af0be22dd1cf3635c2f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:27.332743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103682f-558e-4722-bff5-e1c06160f1fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4807ce933f14af0be22dd1cf3635c2f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:26:27.334628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103682f-558e-4722-bff5-e1c06160f1fb admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:26:27.436304 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f103682f-558e-4722-bff5-e1c06160f1fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4807ce933f14af0be22dd1cf3635c2f', 'request_id': 'req-f103682f-558e-4722-bff5-e1c06160f1fb'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:26:27.442340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f103682f-558e-4722-bff5-e1c06160f1fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4807ce933f14af0be22dd1cf3635c2f', 'request_id': 'req-f103682f-558e-4722-bff5-e1c06160f1fb'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:26:27.527235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f103682f-558e-4722-bff5-e1c06160f1fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4807ce933f14af0be22dd1cf3635c2f', 'request_id': 'req-f103682f-558e-4722-bff5-e1c06160f1fb'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:26:27.537579 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f103682f-558e-4722-bff5-e1c06160f1fb admin admin] The token cache is being invalidate because project c4807ce933f14af0be22dd1cf3635c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:26:27.537862 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f103682f-558e-4722-bff5-e1c06160f1fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:26:27.538513 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f103682f-558e-4722-bff5-e1c06160f1fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:26:27.540081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1381/2733] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:26:27 2022] DELETE /identity/v3/projects/c4807ce933f14af0be22dd1cf3635c2f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:26:27.545880 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3b7ed-f044-4b9f-874b-a2382f3f5631 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:27.546168 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3b7ed-f044-4b9f-874b-a2382f3f5631 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:27.546496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3b7ed-f044-4b9f-874b-a2382f3f5631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:27.564475 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-66b3b7ed-f044-4b9f-874b-a2382f3f5631 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:26:27.565673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-66b3b7ed-f044-4b9f-874b-a2382f3f5631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:26:27.582823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:26:27.627033 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1353/2734] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:26:27.635205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469800e6-13a4-4d8b-9236-56df3e326c2f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:27.635975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469800e6-13a4-4d8b-9236-56df3e326c2f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:27.635975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469800e6-13a4-4d8b-9236-56df3e326c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:27.655368 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-469800e6-13a4-4d8b-9236-56df3e326c2f None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:26:27.656611 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-469800e6-13a4-4d8b-9236-56df3e326c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:26:27.664747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1382/2735] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:26:27.668975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3b7ed-f044-4b9f-874b-a2382f3f5631 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:26:27.678805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b3b7ed-f044-4b9f-874b-a2382f3f5631 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:26:27.683425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f9b04-56c8-44aa-93a5-3e7aa5f602cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:26:27.684076 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f9b04-56c8-44aa-93a5-3e7aa5f602cc admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:27.684356 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f9b04-56c8-44aa-93a5-3e7aa5f602cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:27.684650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f9b04-56c8-44aa-93a5-3e7aa5f602cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:27.685348 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f9b04-56c8-44aa-93a5-3e7aa5f602cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:26:27.685652 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f9b04-56c8-44aa-93a5-3e7aa5f602cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:26:27.687243 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f9b04-56c8-44aa-93a5-3e7aa5f602cc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:26:27.699140 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1354/2736] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:26:27.703299 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469800e6-13a4-4d8b-9236-56df3e326c2f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:26:27.712482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469800e6-13a4-4d8b-9236-56df3e326c2f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:26:27.717800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34278f10-2e96-46fd-8fe1-588d54b3dd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:26:27.718443 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34278f10-2e96-46fd-8fe1-588d54b3dd89 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:27.718728 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34278f10-2e96-46fd-8fe1-588d54b3dd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:27.719004 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34278f10-2e96-46fd-8fe1-588d54b3dd89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:27.719734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34278f10-2e96-46fd-8fe1-588d54b3dd89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:26:27.721622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34278f10-2e96-46fd-8fe1-588d54b3dd89 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:26:27.731871 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:26:27.802688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1383/2737] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:26:27 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:26:27.807691 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f9b04-56c8-44aa-93a5-3e7aa5f602cc admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:26:27.817643 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f9b04-56c8-44aa-93a5-3e7aa5f602cc admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:26:27.876244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb4fb1-3226-49de-bec5-1ae5639a1b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:26:27.876911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb4fb1-3226-49de-bec5-1ae5639a1b40 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:27.877192 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb4fb1-3226-49de-bec5-1ae5639a1b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:27.877463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb4fb1-3226-49de-bec5-1ae5639a1b40 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:27.878153 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb4fb1-3226-49de-bec5-1ae5639a1b40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:26:27.880085 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb4fb1-3226-49de-bec5-1ae5639a1b40 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:26:27.984887 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1355/2738] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:26:27 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:26:27.989138 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34278f10-2e96-46fd-8fe1-588d54b3dd89 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:26:28.000076 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34278f10-2e96-46fd-8fe1-588d54b3dd89 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:26:28.004942 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd00898-11dd-4832-aa9f-efadaeaab63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:26:28.005604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd00898-11dd-4832-aa9f-efadaeaab63c admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:28.005936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd00898-11dd-4832-aa9f-efadaeaab63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:28.006244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd00898-11dd-4832-aa9f-efadaeaab63c admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:28.006975 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd00898-11dd-4832-aa9f-efadaeaab63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:26:28.008710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd00898-11dd-4832-aa9f-efadaeaab63c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:26:28.019859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1384/2739] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:26:28.024234 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb4fb1-3226-49de-bec5-1ae5639a1b40 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:26:28.034100 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb4fb1-3226-49de-bec5-1ae5639a1b40 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:26:28.038802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154287d5-4cb1-4523-9124-484bc7a84a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:26:28.039486 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154287d5-4cb1-4523-9124-484bc7a84a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:28.039800 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154287d5-4cb1-4523-9124-484bc7a84a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:28.040082 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154287d5-4cb1-4523-9124-484bc7a84a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:28.040803 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154287d5-4cb1-4523-9124-484bc7a84a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:26:28.042644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154287d5-4cb1-4523-9124-484bc7a84a41 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:26:28.052575 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1356/2740] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:26:28.057258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd00898-11dd-4832-aa9f-efadaeaab63c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:26:28.067274 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd00898-11dd-4832-aa9f-efadaeaab63c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:26:28.071915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df321ea3-f751-45d5-8496-8fa70a8613dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:26:28.073636 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df321ea3-f751-45d5-8496-8fa70a8613dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:28.073925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df321ea3-f751-45d5-8496-8fa70a8613dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:28.074207 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df321ea3-f751-45d5-8496-8fa70a8613dc admin admin] PATH_INFO: `/v3/projects/9c5f325c1c3245fa88dbb36698037138/users/f418500a1fa84bff9e3b41c45a0bc643/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:28.088713 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df321ea3-f751-45d5-8496-8fa70a8613dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c5f325c1c3245fa88dbb36698037138, user_id=f418500a1fa84bff9e3b41c45a0bc643, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:26:28.091414 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df321ea3-f751-45d5-8496-8fa70a8613dc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:26:28.138711 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1385/2741] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:26:28 2022] PUT /identity/v3/projects/9c5f325c1c3245fa88dbb36698037138/users/f418500a1fa84bff9e3b41c45a0bc643/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:26:28.143874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f08bc-2974-469a-a492-e94a83e0bd52 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:28.144169 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f08bc-2974-469a-a492-e94a83e0bd52 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:28.144452 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f08bc-2974-469a-a492-e94a83e0bd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:28.162589 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-df8f08bc-2974-469a-a492-e94a83e0bd52 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:26:28.163826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-df8f08bc-2974-469a-a492-e94a83e0bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:26:28.223056 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1357/2742] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:26:28.232726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df321ea3-f751-45d5-8496-8fa70a8613dc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:26:28.234417 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-df321ea3-f751-45d5-8496-8fa70a8613dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:26:28.291909 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df321ea3-f751-45d5-8496-8fa70a8613dc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:26:28.295804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947289e1-afa1-40dd-807d-bc0c43d494d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:26:28.296454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947289e1-afa1-40dd-807d-bc0c43d494d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:28.296741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947289e1-afa1-40dd-807d-bc0c43d494d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:28.297022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947289e1-afa1-40dd-807d-bc0c43d494d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:28.306914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947289e1-afa1-40dd-807d-bc0c43d494d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:26:28.308476 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947289e1-afa1-40dd-807d-bc0c43d494d3 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:26:28.317842 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1386/2743] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:26:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:26:33.403435 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37478e41-6c53-4ad3-bc14-a24014ea193b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:33.403981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37478e41-6c53-4ad3-bc14-a24014ea193b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:33.403981 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37478e41-6c53-4ad3-bc14-a24014ea193b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:33.433929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-37478e41-6c53-4ad3-bc14-a24014ea193b None None] MFA Rules not processed for user `f418500a1fa84bff9e3b41c45a0bc643`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:26:33.435423 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-37478e41-6c53-4ad3-bc14-a24014ea193b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:26:33.454222 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:26:33.518827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1358/2744] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:26:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:26:33.529439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947289e1-afa1-40dd-807d-bc0c43d494d3 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:26:33.531052 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-947289e1-afa1-40dd-807d-bc0c43d494d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:26:33.607872 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947289e1-afa1-40dd-807d-bc0c43d494d3 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:26:33.613390 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a55cb-c7db-42e2-8aa7-ee3fa25cc820 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:26:33.614045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a55cb-c7db-42e2-8aa7-ee3fa25cc820 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:33.614324 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a55cb-c7db-42e2-8aa7-ee3fa25cc820 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:33.614597 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a55cb-c7db-42e2-8aa7-ee3fa25cc820 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:33.623624 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a55cb-c7db-42e2-8aa7-ee3fa25cc820 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:26:33.625236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a55cb-c7db-42e2-8aa7-ee3fa25cc820 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:26:33.634155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1387/2745] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:26:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:26:35.951089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48da70-5368-4796-a3da-a42a443d822b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:26:35.951715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48da70-5368-4796-a3da-a42a443d822b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:26:35.951715 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48da70-5368-4796-a3da-a42a443d822b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:26:35.970360 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6f48da70-5368-4796-a3da-a42a443d822b None None] MFA Rules not processed for user `f418500a1fa84bff9e3b41c45a0bc643`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:26:35.971576 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6f48da70-5368-4796-a3da-a42a443d822b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:26:35.978780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1359/2746] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:26:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:26:43.630210 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=78,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:26:45.963994 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=22 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:27:01.792082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a55cb-c7db-42e2-8aa7-ee3fa25cc820 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:27:01.793535 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ed9a55cb-c7db-42e2-8aa7-ee3fa25cc820 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:27:01.806155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a55cb-c7db-42e2-8aa7-ee3fa25cc820 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:27:01.809800 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb8a74-83ff-44ef-a6d6-773ed2d92531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:27:01.810439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb8a74-83ff-44ef-a6d6-773ed2d92531 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:27:01.810720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb8a74-83ff-44ef-a6d6-773ed2d92531 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:27:01.810992 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb8a74-83ff-44ef-a6d6-773ed2d92531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:27:01.813007 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-77cb8a74-83ff-44ef-a6d6-773ed2d92531 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:27:01.873525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb8a74-83ff-44ef-a6d6-773ed2d92531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:27:01.875718 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb8a74-83ff-44ef-a6d6-773ed2d92531 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:27:01.885022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1388/2747] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:27:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:27:06.898557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48da70-5368-4796-a3da-a42a443d822b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:27:06.900971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6f48da70-5368-4796-a3da-a42a443d822b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:27:06.978132 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb8a74-83ff-44ef-a6d6-773ed2d92531 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:27:06.980194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-77cb8a74-83ff-44ef-a6d6-773ed2d92531 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:27:07.019310 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48da70-5368-4796-a3da-a42a443d822b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:27:07.025802 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58533d-bf61-4d7e-87d1-109de4cf2bbc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:27:07.026585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58533d-bf61-4d7e-87d1-109de4cf2bbc service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:27:07.026877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58533d-bf61-4d7e-87d1-109de4cf2bbc service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:27:07.027155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58533d-bf61-4d7e-87d1-109de4cf2bbc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:27:07.029538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bd58533d-bf61-4d7e-87d1-109de4cf2bbc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:27:07.044593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58533d-bf61-4d7e-87d1-109de4cf2bbc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:27:07.046646 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58533d-bf61-4d7e-87d1-109de4cf2bbc service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:27:07.058280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1360/2748] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:27:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:27:07.060809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb8a74-83ff-44ef-a6d6-773ed2d92531 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:27:07.066261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcb81c-ff0e-4f6f-8a4a-d05238717195 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:27:07.075482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcb81c-ff0e-4f6f-8a4a-d05238717195 service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:27:07.075482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcb81c-ff0e-4f6f-8a4a-d05238717195 service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:27:07.075482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcb81c-ff0e-4f6f-8a4a-d05238717195 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:27:07.076914 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bcb81c-ff0e-4f6f-8a4a-d05238717195 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:27:07.078673 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bcb81c-ff0e-4f6f-8a4a-d05238717195 service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:27:07.095050 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1389/2749] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:27:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:27:08.347008 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58533d-bf61-4d7e-87d1-109de4cf2bbc service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:27:08.356825 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58533d-bf61-4d7e-87d1-109de4cf2bbc service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:27:08.360665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17056957-d52c-4e98-bf3c-50acb332c04e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:27:08.364279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17056957-d52c-4e98-bf3c-50acb332c04e service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:27:08.364279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17056957-d52c-4e98-bf3c-50acb332c04e service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:27:08.364279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17056957-d52c-4e98-bf3c-50acb332c04e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:27:08.364279 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-17056957-d52c-4e98-bf3c-50acb332c04e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:27:08.376180 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17056957-d52c-4e98-bf3c-50acb332c04e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:27:08.377608 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17056957-d52c-4e98-bf3c-50acb332c04e service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:27:08.387247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1361/2750] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:27:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:27:09.095041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471d396-9ffe-4732-b65a-4586b59a9873 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:27:09.095041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471d396-9ffe-4732-b65a-4586b59a9873 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:27:09.095041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471d396-9ffe-4732-b65a-4586b59a9873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:27:09.112460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1471d396-9ffe-4732-b65a-4586b59a9873 None None] MFA Rules not processed for user `6f9c73ca3c364117993dac93acf5a737`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:27:09.113738 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1471d396-9ffe-4732-b65a-4586b59a9873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:27:09.170978 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1390/2751] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:27:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:27:16.308740 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17056957-d52c-4e98-bf3c-50acb332c04e service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:27:16.310367 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-17056957-d52c-4e98-bf3c-50acb332c04e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:27:16.323272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17056957-d52c-4e98-bf3c-50acb332c04e service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:27:16.328158 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd5f59-5233-47fd-b5b3-f880767d9e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:27:16.329037 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd5f59-5233-47fd-b5b3-f880767d9e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:27:16.329320 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd5f59-5233-47fd-b5b3-f880767d9e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:27:16.330219 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd5f59-5233-47fd-b5b3-f880767d9e80 admin admin] PATH_INFO: `/v3/users/bb55f38fa3c741cbb7f00e310368a85b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:27:16.331470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd5f59-5233-47fd-b5b3-f880767d9e80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb55f38fa3c741cbb7f00e310368a85b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:27:16.333439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd5f59-5233-47fd-b5b3-f880767d9e80 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:27:16.557852 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fcdd5f59-5233-47fd-b5b3-f880767d9e80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb55f38fa3c741cbb7f00e310368a85b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:16.570121 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fcdd5f59-5233-47fd-b5b3-f880767d9e80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb55f38fa3c741cbb7f00e310368a85b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:16.570793 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fcdd5f59-5233-47fd-b5b3-f880767d9e80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb55f38fa3c741cbb7f00e310368a85b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:16.577571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fcdd5f59-5233-47fd-b5b3-f880767d9e80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb55f38fa3c741cbb7f00e310368a85b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:16.578226 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fcdd5f59-5233-47fd-b5b3-f880767d9e80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb55f38fa3c741cbb7f00e310368a85b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:16.646789 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1362/2752] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:27:16 2022] DELETE /identity/v3/users/bb55f38fa3c741cbb7f00e310368a85b => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:27:16.900751 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471d396-9ffe-4732-b65a-4586b59a9873 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:27:16.912762 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471d396-9ffe-4732-b65a-4586b59a9873 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:27:16.946606 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:27:16.971643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4068888-1c62-461d-9f03-2eebf7a30ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:27:16.972504 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4068888-1c62-461d-9f03-2eebf7a30ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:27:16.972792 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4068888-1c62-461d-9f03-2eebf7a30ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:27:16.973066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4068888-1c62-461d-9f03-2eebf7a30ae8 admin admin] PATH_INFO: `/v3/projects/66edbd777b48472b8fcd5a3f43c68dcc` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:27:16.974240 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4068888-1c62-461d-9f03-2eebf7a30ae8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66edbd777b48472b8fcd5a3f43c68dcc)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:27:16.976305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4068888-1c62-461d-9f03-2eebf7a30ae8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:27:17.110386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d4068888-1c62-461d-9f03-2eebf7a30ae8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66edbd777b48472b8fcd5a3f43c68dcc', 'request_id': 'req-d4068888-1c62-461d-9f03-2eebf7a30ae8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:17.116547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d4068888-1c62-461d-9f03-2eebf7a30ae8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66edbd777b48472b8fcd5a3f43c68dcc', 'request_id': 'req-d4068888-1c62-461d-9f03-2eebf7a30ae8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:17.183353 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d4068888-1c62-461d-9f03-2eebf7a30ae8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66edbd777b48472b8fcd5a3f43c68dcc', 'request_id': 'req-d4068888-1c62-461d-9f03-2eebf7a30ae8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:17.193852 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d4068888-1c62-461d-9f03-2eebf7a30ae8 admin admin] The token cache is being invalidate because project 66edbd777b48472b8fcd5a3f43c68dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:27:17.194142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d4068888-1c62-461d-9f03-2eebf7a30ae8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:17.194767 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d4068888-1c62-461d-9f03-2eebf7a30ae8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:17.196347 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1391/2753] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:27:16 2022] DELETE /identity/v3/projects/66edbd777b48472b8fcd5a3f43c68dcc => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:27:17.200446 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd5f59-5233-47fd-b5b3-f880767d9e80 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:27:17.201856 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fcdd5f59-5233-47fd-b5b3-f880767d9e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:27:17.215631 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd5f59-5233-47fd-b5b3-f880767d9e80 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:27:17.259983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:27:17.271863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca68fc3-cf7f-4611-9205-986bdc9fd155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:27:17.273478 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca68fc3-cf7f-4611-9205-986bdc9fd155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:27:17.273478 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca68fc3-cf7f-4611-9205-986bdc9fd155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:27:17.273693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca68fc3-cf7f-4611-9205-986bdc9fd155 admin admin] PATH_INFO: `/v3/users/d6ab2a6d480c4de1adb0de26132d1ba3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:27:17.274678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca68fc3-cf7f-4611-9205-986bdc9fd155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6ab2a6d480c4de1adb0de26132d1ba3)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:27:17.276606 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca68fc3-cf7f-4611-9205-986bdc9fd155 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:27:17.419001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ca68fc3-cf7f-4611-9205-986bdc9fd155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6ab2a6d480c4de1adb0de26132d1ba3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:17.430803 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ca68fc3-cf7f-4611-9205-986bdc9fd155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6ab2a6d480c4de1adb0de26132d1ba3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:17.431493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ca68fc3-cf7f-4611-9205-986bdc9fd155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6ab2a6d480c4de1adb0de26132d1ba3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:17.439908 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ca68fc3-cf7f-4611-9205-986bdc9fd155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6ab2a6d480c4de1adb0de26132d1ba3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:17.440555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ca68fc3-cf7f-4611-9205-986bdc9fd155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6ab2a6d480c4de1adb0de26132d1ba3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:17.524119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1363/2754] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:27:17 2022] DELETE /identity/v3/users/d6ab2a6d480c4de1adb0de26132d1ba3 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:27:17.845626 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4068888-1c62-461d-9f03-2eebf7a30ae8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:27:17.847088 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d4068888-1c62-461d-9f03-2eebf7a30ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:27:17.863710 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4068888-1c62-461d-9f03-2eebf7a30ae8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:27:17.922479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abae300-aa8e-4a83-a65e-8e92e95d1940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:27:17.923399 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abae300-aa8e-4a83-a65e-8e92e95d1940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:27:17.923684 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abae300-aa8e-4a83-a65e-8e92e95d1940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:27:17.923989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abae300-aa8e-4a83-a65e-8e92e95d1940 admin admin] PATH_INFO: `/v3/projects/56498dbba20e41f5a5b6b04e6d21a39b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:27:17.925175 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abae300-aa8e-4a83-a65e-8e92e95d1940 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56498dbba20e41f5a5b6b04e6d21a39b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:27:17.927101 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abae300-aa8e-4a83-a65e-8e92e95d1940 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:27:18.028901 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7abae300-aa8e-4a83-a65e-8e92e95d1940 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56498dbba20e41f5a5b6b04e6d21a39b', 'request_id': 'req-7abae300-aa8e-4a83-a65e-8e92e95d1940'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:18.034843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7abae300-aa8e-4a83-a65e-8e92e95d1940 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56498dbba20e41f5a5b6b04e6d21a39b', 'request_id': 'req-7abae300-aa8e-4a83-a65e-8e92e95d1940'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:18.101583 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7abae300-aa8e-4a83-a65e-8e92e95d1940 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56498dbba20e41f5a5b6b04e6d21a39b', 'request_id': 'req-7abae300-aa8e-4a83-a65e-8e92e95d1940'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:18.105797 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:27:18.113229 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7abae300-aa8e-4a83-a65e-8e92e95d1940 admin admin] The token cache is being invalidate because project 56498dbba20e41f5a5b6b04e6d21a39b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:27:18.116925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7abae300-aa8e-4a83-a65e-8e92e95d1940 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:18.116925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7abae300-aa8e-4a83-a65e-8e92e95d1940 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:27:18.116925 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1392/2755] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:27:17 2022] DELETE /identity/v3/projects/56498dbba20e41f5a5b6b04e6d21a39b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:27:27.521295 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:27:28.249848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:28:32.444939 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca68fc3-cf7f-4611-9205-986bdc9fd155 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:28:32.446352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5ca68fc3-cf7f-4611-9205-986bdc9fd155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:28:32.502145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca68fc3-cf7f-4611-9205-986bdc9fd155 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:28:32.549377 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dea3d1-e56d-4a34-be79-ee79f6fa2ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:28:32.550020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dea3d1-e56d-4a34-be79-ee79f6fa2ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:28:32.550338 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dea3d1-e56d-4a34-be79-ee79f6fa2ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:28:32.550617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dea3d1-e56d-4a34-be79-ee79f6fa2ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:28:32.552431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-65dea3d1-e56d-4a34-be79-ee79f6fa2ad7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:28:32.620950 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dea3d1-e56d-4a34-be79-ee79f6fa2ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:28:32.622326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dea3d1-e56d-4a34-be79-ee79f6fa2ad7 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:28:32.633598 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1364/2756] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:28:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:28:36.077899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abae300-aa8e-4a83-a65e-8e92e95d1940 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:28:36.079409 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7abae300-aa8e-4a83-a65e-8e92e95d1940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:28:36.092536 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abae300-aa8e-4a83-a65e-8e92e95d1940 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:28:36.096134 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4178b-ce3d-4880-8147-00dce5d926b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:28:36.096771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4178b-ce3d-4880-8147-00dce5d926b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:28:36.097110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4178b-ce3d-4880-8147-00dce5d926b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:28:36.097408 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4178b-ce3d-4880-8147-00dce5d926b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:28:36.099383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-06c4178b-ce3d-4880-8147-00dce5d926b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:28:36.112729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4178b-ce3d-4880-8147-00dce5d926b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:28:36.114150 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4178b-ce3d-4880-8147-00dce5d926b0 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:28:36.126660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1393/2757] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:28:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:28:42.628465 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=74 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:28:46.121189 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:29:12.664238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dea3d1-e56d-4a34-be79-ee79f6fa2ad7 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:29:12.665664 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-65dea3d1-e56d-4a34-be79-ee79f6fa2ad7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:29:12.722832 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dea3d1-e56d-4a34-be79-ee79f6fa2ad7 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:29:12.726369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1095da-7caa-48b9-9b9e-b74e8c98a880 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:29:12.727046 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1095da-7caa-48b9-9b9e-b74e8c98a880 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:29:12.727366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1095da-7caa-48b9-9b9e-b74e8c98a880 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:29:12.727647 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1095da-7caa-48b9-9b9e-b74e8c98a880 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:29:12.729419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-de1095da-7caa-48b9-9b9e-b74e8c98a880 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:29:12.788359 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1095da-7caa-48b9-9b9e-b74e8c98a880 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:29:12.789789 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1095da-7caa-48b9-9b9e-b74e8c98a880 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:29:12.798819 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1365/2758] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:29:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:29:12.900491 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6aa2e-438c-470e-a641-e3611fee000a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:29:12.900995 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6aa2e-438c-470e-a641-e3611fee000a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:29:12.901104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6aa2e-438c-470e-a641-e3611fee000a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:29:12.919357 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2ed6aa2e-438c-470e-a641-e3611fee000a None None] MFA Rules not processed for user `6f9c73ca3c364117993dac93acf5a737`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:29:12.920562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2ed6aa2e-438c-470e-a641-e3611fee000a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:29:12.928009 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1394/2759] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:29:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:29:16.882215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1095da-7caa-48b9-9b9e-b74e8c98a880 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:29:16.891436 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1095da-7caa-48b9-9b9e-b74e8c98a880 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:29:16.896329 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed93d2c-a32a-409f-844b-5969eb3bbe2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:29:16.896979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed93d2c-a32a-409f-844b-5969eb3bbe2d service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:29:16.897266 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed93d2c-a32a-409f-844b-5969eb3bbe2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:29:16.897540 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed93d2c-a32a-409f-844b-5969eb3bbe2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:29:16.899343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8ed93d2c-a32a-409f-844b-5969eb3bbe2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:29:16.909730 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed93d2c-a32a-409f-844b-5969eb3bbe2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:29:16.911141 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed93d2c-a32a-409f-844b-5969eb3bbe2d service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:29:16.919142 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1366/2760] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:29:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:29:22.914416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:29:25.925543 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f3627-8427-42e9-b653-904c6ea7dcd7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:29:25.926148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f3627-8427-42e9-b653-904c6ea7dcd7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:29:25.926148 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f3627-8427-42e9-b653-904c6ea7dcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:29:25.943487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7d9f3627-8427-42e9-b653-904c6ea7dcd7 None None] MFA Rules not processed for user `f418500a1fa84bff9e3b41c45a0bc643`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:29:25.944688 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7d9f3627-8427-42e9-b653-904c6ea7dcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:29:26.002687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1395/2761] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:29:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:29:26.915480 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=54 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:29:35.991647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=25 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:30:41.813538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed93d2c-a32a-409f-844b-5969eb3bbe2d service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:41.815618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8ed93d2c-a32a-409f-844b-5969eb3bbe2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:41.836146 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed93d2c-a32a-409f-844b-5969eb3bbe2d service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:41.841918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333888e-63fc-4a0b-8c93-228e7a9bf115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:41.842593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333888e-63fc-4a0b-8c93-228e7a9bf115 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:41.842876 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333888e-63fc-4a0b-8c93-228e7a9bf115 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:41.843152 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333888e-63fc-4a0b-8c93-228e7a9bf115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:41.845878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e333888e-63fc-4a0b-8c93-228e7a9bf115 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:41.860054 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333888e-63fc-4a0b-8c93-228e7a9bf115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:41.861757 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333888e-63fc-4a0b-8c93-228e7a9bf115 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:41.875557 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1367/2762] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:30:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:30:51.868913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=6 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:30:52.211109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f3627-8427-42e9-b653-904c6ea7dcd7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:52.216460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7d9f3627-8427-42e9-b653-904c6ea7dcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:52.279301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f3627-8427-42e9-b653-904c6ea7dcd7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:52.283951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb01e97-927e-4ca5-ab9b-9a6ccb351a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:52.284823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb01e97-927e-4ca5-ab9b-9a6ccb351a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:52.285107 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb01e97-927e-4ca5-ab9b-9a6ccb351a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:52.285385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb01e97-927e-4ca5-ab9b-9a6ccb351a34 admin admin] PATH_INFO: `/v3/users/f418500a1fa84bff9e3b41c45a0bc643` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:52.286569 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb01e97-927e-4ca5-ab9b-9a6ccb351a34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f418500a1fa84bff9e3b41c45a0bc643)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:52.288539 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb01e97-927e-4ca5-ab9b-9a6ccb351a34 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:52.499121 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb01e97-927e-4ca5-ab9b-9a6ccb351a34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f418500a1fa84bff9e3b41c45a0bc643'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:52.511374 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb01e97-927e-4ca5-ab9b-9a6ccb351a34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f418500a1fa84bff9e3b41c45a0bc643'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:52.573825 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb01e97-927e-4ca5-ab9b-9a6ccb351a34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f418500a1fa84bff9e3b41c45a0bc643'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:52.574456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb01e97-927e-4ca5-ab9b-9a6ccb351a34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f418500a1fa84bff9e3b41c45a0bc643'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:52.582298 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb01e97-927e-4ca5-ab9b-9a6ccb351a34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f418500a1fa84bff9e3b41c45a0bc643'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:52.588824 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1396/2763] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:30:52 2022] DELETE /identity/v3/users/f418500a1fa84bff9e3b41c45a0bc643 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:30:52.845093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333888e-63fc-4a0b-8c93-228e7a9bf115 service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:52.856807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333888e-63fc-4a0b-8c93-228e7a9bf115 service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:52.919445 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13610f0-e585-4c9d-bd0d-bb7f31466f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:52.922986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13610f0-e585-4c9d-bd0d-bb7f31466f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:52.922986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13610f0-e585-4c9d-bd0d-bb7f31466f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:52.922986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13610f0-e585-4c9d-bd0d-bb7f31466f8b admin admin] PATH_INFO: `/v3/projects/9c5f325c1c3245fa88dbb36698037138` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:52.922986 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13610f0-e585-4c9d-bd0d-bb7f31466f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c5f325c1c3245fa88dbb36698037138)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:52.924006 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13610f0-e585-4c9d-bd0d-bb7f31466f8b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:53.002342 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c13610f0-e585-4c9d-bd0d-bb7f31466f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c5f325c1c3245fa88dbb36698037138', 'request_id': 'req-c13610f0-e585-4c9d-bd0d-bb7f31466f8b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:53.008513 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c13610f0-e585-4c9d-bd0d-bb7f31466f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c5f325c1c3245fa88dbb36698037138', 'request_id': 'req-c13610f0-e585-4c9d-bd0d-bb7f31466f8b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:53.053617 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c13610f0-e585-4c9d-bd0d-bb7f31466f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c5f325c1c3245fa88dbb36698037138', 'request_id': 'req-c13610f0-e585-4c9d-bd0d-bb7f31466f8b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:53.064729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c13610f0-e585-4c9d-bd0d-bb7f31466f8b admin admin] The token cache is being invalidate because project 9c5f325c1c3245fa88dbb36698037138 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:30:53.064729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c13610f0-e585-4c9d-bd0d-bb7f31466f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:53.066444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c13610f0-e585-4c9d-bd0d-bb7f31466f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:53.067288 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1368/2764] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:30:52 2022] DELETE /identity/v3/projects/9c5f325c1c3245fa88dbb36698037138 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:30:53.072428 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7ce9f-b9c6-443a-b53d-3d32de3e65cd None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:53.072722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7ce9f-b9c6-443a-b53d-3d32de3e65cd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:53.073003 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7ce9f-b9c6-443a-b53d-3d32de3e65cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:53.096254 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2db7ce9f-b9c6-443a-b53d-3d32de3e65cd None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:30:53.097472 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2db7ce9f-b9c6-443a-b53d-3d32de3e65cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:53.156503 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1397/2765] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:30:53.164524 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8daba-ee99-4d55-96f9-806bbda80f17 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:53.164818 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8daba-ee99-4d55-96f9-806bbda80f17 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:53.165097 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8daba-ee99-4d55-96f9-806bbda80f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:53.183086 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-58c8daba-ee99-4d55-96f9-806bbda80f17 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:30:53.184407 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-58c8daba-ee99-4d55-96f9-806bbda80f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:53.192953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1369/2766] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:30:53.197217 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7ce9f-b9c6-443a-b53d-3d32de3e65cd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:53.207731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7ce9f-b9c6-443a-b53d-3d32de3e65cd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:53.212355 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f045296-d0a3-405d-9faf-3b57f66de34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:53.213006 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f045296-d0a3-405d-9faf-3b57f66de34b admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:53.213291 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f045296-d0a3-405d-9faf-3b57f66de34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:53.213601 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f045296-d0a3-405d-9faf-3b57f66de34b admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:53.214322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f045296-d0a3-405d-9faf-3b57f66de34b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:30:53.214632 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f045296-d0a3-405d-9faf-3b57f66de34b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:53.216258 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f045296-d0a3-405d-9faf-3b57f66de34b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:53.228371 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1398/2767] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:30:53.232833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8daba-ee99-4d55-96f9-806bbda80f17 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:53.241979 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8daba-ee99-4d55-96f9-806bbda80f17 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:53.246541 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e207361-421c-4801-ac1f-3b6d7dca7a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:53.247207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e207361-421c-4801-ac1f-3b6d7dca7a25 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:53.247497 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e207361-421c-4801-ac1f-3b6d7dca7a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:53.247792 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e207361-421c-4801-ac1f-3b6d7dca7a25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:53.249005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e207361-421c-4801-ac1f-3b6d7dca7a25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:53.251019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e207361-421c-4801-ac1f-3b6d7dca7a25 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:53.312693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1370/2768] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:30:53 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:30:53.316807 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f045296-d0a3-405d-9faf-3b57f66de34b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:53.327804 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f045296-d0a3-405d-9faf-3b57f66de34b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:53.331418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:30:53.386656 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa7da1-d3fb-4c77-b169-bc6b1c80a78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:53.389724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa7da1-d3fb-4c77-b169-bc6b1c80a78b admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:53.389724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa7da1-d3fb-4c77-b169-bc6b1c80a78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:53.389724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa7da1-d3fb-4c77-b169-bc6b1c80a78b admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:53.389724 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa7da1-d3fb-4c77-b169-bc6b1c80a78b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:53.391113 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa7da1-d3fb-4c77-b169-bc6b1c80a78b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:53.494385 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1399/2769] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:30:53 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:30:53.500572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e207361-421c-4801-ac1f-3b6d7dca7a25 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:53.517126 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e207361-421c-4801-ac1f-3b6d7dca7a25 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:53.523820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e33f1-3838-4ada-81f8-390fce07f35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:53.524504 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e33f1-3838-4ada-81f8-390fce07f35a admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:53.524856 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e33f1-3838-4ada-81f8-390fce07f35a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:53.525357 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e33f1-3838-4ada-81f8-390fce07f35a admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:53.526111 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e33f1-3838-4ada-81f8-390fce07f35a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:53.527742 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e33f1-3838-4ada-81f8-390fce07f35a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:53.539005 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1371/2770] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:30:53.543205 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa7da1-d3fb-4c77-b169-bc6b1c80a78b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:53.554125 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa7da1-d3fb-4c77-b169-bc6b1c80a78b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:53.558956 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96319f5-3a82-4800-b38d-4a544647a0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:53.559665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96319f5-3a82-4800-b38d-4a544647a0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:53.559952 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96319f5-3a82-4800-b38d-4a544647a0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:53.560230 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96319f5-3a82-4800-b38d-4a544647a0c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:53.561834 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96319f5-3a82-4800-b38d-4a544647a0c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:53.563622 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96319f5-3a82-4800-b38d-4a544647a0c8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:53.575228 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1400/2771] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:30:53.578877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e33f1-3838-4ada-81f8-390fce07f35a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:53.589565 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e33f1-3838-4ada-81f8-390fce07f35a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:53.594641 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e2057-c028-4d89-9f0e-594511d84b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:53.595558 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e2057-c028-4d89-9f0e-594511d84b5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:53.595840 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e2057-c028-4d89-9f0e-594511d84b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:53.596113 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e2057-c028-4d89-9f0e-594511d84b5e admin admin] PATH_INFO: `/v3/projects/9c0e7447465d430d9352d9e9b7ea012c/users/920e7c03e9b94550880b452d84f07128/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:53.612057 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e2057-c028-4d89-9f0e-594511d84b5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c0e7447465d430d9352d9e9b7ea012c, user_id=920e7c03e9b94550880b452d84f07128, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:53.614758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e2057-c028-4d89-9f0e-594511d84b5e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:53.722795 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1372/2772] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:30:53 2022] PUT /identity/v3/projects/9c0e7447465d430d9352d9e9b7ea012c/users/920e7c03e9b94550880b452d84f07128/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:30:53.728063 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5903e3-e192-4a9f-82c8-7b3fdf4c2e40 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:53.728360 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5903e3-e192-4a9f-82c8-7b3fdf4c2e40 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:53.728647 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5903e3-e192-4a9f-82c8-7b3fdf4c2e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:53.747424 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ed5903e3-e192-4a9f-82c8-7b3fdf4c2e40 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:30:53.749287 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ed5903e3-e192-4a9f-82c8-7b3fdf4c2e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:53.812105 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1401/2773] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:30:53.821778 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e2057-c028-4d89-9f0e-594511d84b5e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:53.823597 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a26e2057-c028-4d89-9f0e-594511d84b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:53.858254 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:30:53.886801 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e2057-c028-4d89-9f0e-594511d84b5e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:53.891849 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc95f7-ab15-4476-a3fc-9dac38d9e5d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:53.892512 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc95f7-ab15-4476-a3fc-9dac38d9e5d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:53.892790 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc95f7-ab15-4476-a3fc-9dac38d9e5d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:53.893064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc95f7-ab15-4476-a3fc-9dac38d9e5d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:53.902775 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc95f7-ab15-4476-a3fc-9dac38d9e5d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:53.904216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc95f7-ab15-4476-a3fc-9dac38d9e5d5 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:53.914414 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1373/2774] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:30:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:30:59.502470 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97bfa6-4679-456b-b345-6b296d8de981 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:59.502980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97bfa6-4679-456b-b345-6b296d8de981 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:59.503080 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97bfa6-4679-456b-b345-6b296d8de981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:59.571478 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fc97bfa6-4679-456b-b345-6b296d8de981 None None] MFA Rules not processed for user `920e7c03e9b94550880b452d84f07128`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:30:59.573186 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fc97bfa6-4679-456b-b345-6b296d8de981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:59.664579 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:30:59.683219 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1402/2775] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:30:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:30:59.697957 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc95f7-ab15-4476-a3fc-9dac38d9e5d5 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:59.699829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9dfc95f7-ab15-4476-a3fc-9dac38d9e5d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:59.837547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc95f7-ab15-4476-a3fc-9dac38d9e5d5 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:59.843975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc87b71-ed45-4999-932a-f04fddeae94d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:59.844912 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc87b71-ed45-4999-932a-f04fddeae94d service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:59.845277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc87b71-ed45-4999-932a-f04fddeae94d service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:59.845624 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc87b71-ed45-4999-932a-f04fddeae94d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:59.854795 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc87b71-ed45-4999-932a-f04fddeae94d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:59.857091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc87b71-ed45-4999-932a-f04fddeae94d service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:59.867769 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1374/2776] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:30:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:31:02.416363 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e5f2-041c-4fea-aa6d-59feec27efe1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:31:02.416839 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e5f2-041c-4fea-aa6d-59feec27efe1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:31:02.417041 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e5f2-041c-4fea-aa6d-59feec27efe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:31:02.459309 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6475e5f2-041c-4fea-aa6d-59feec27efe1 None None] MFA Rules not processed for user `920e7c03e9b94550880b452d84f07128`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:31:02.459309 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6475e5f2-041c-4fea-aa6d-59feec27efe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:31:02.468583 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1403/2777] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:31:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:31:02.943611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc87b71-ed45-4999-932a-f04fddeae94d service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:31:02.948476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ebc87b71-ed45-4999-932a-f04fddeae94d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:31:02.974220 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc87b71-ed45-4999-932a-f04fddeae94d service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:31:02.978546 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f67a2-137a-4501-bdfe-949d53cc3f5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:31:02.979593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f67a2-137a-4501-bdfe-949d53cc3f5c service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:31:02.980102 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f67a2-137a-4501-bdfe-949d53cc3f5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:31:02.980476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f67a2-137a-4501-bdfe-949d53cc3f5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:31:02.983933 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-db9f67a2-137a-4501-bdfe-949d53cc3f5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:31:03.006093 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f67a2-137a-4501-bdfe-949d53cc3f5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:31:03.007399 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f67a2-137a-4501-bdfe-949d53cc3f5c service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:31:03.033745 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1375/2778] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:31:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:31:03.487108 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475e5f2-041c-4fea-aa6d-59feec27efe1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:31:03.488729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6475e5f2-041c-4fea-aa6d-59feec27efe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:31:03.589743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475e5f2-041c-4fea-aa6d-59feec27efe1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:31:03.595667 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa37ad4-76b1-4008-812b-7b4bd85e10af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:31:03.596635 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa37ad4-76b1-4008-812b-7b4bd85e10af service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:31:03.597124 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa37ad4-76b1-4008-812b-7b4bd85e10af service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:31:03.597479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa37ad4-76b1-4008-812b-7b4bd85e10af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:31:03.599590 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-baa37ad4-76b1-4008-812b-7b4bd85e10af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:31:03.614894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa37ad4-76b1-4008-812b-7b4bd85e10af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:31:03.616782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa37ad4-76b1-4008-812b-7b4bd85e10af service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:31:03.629522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1404/2779] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:31:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:31:12.524855 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f67a2-137a-4501-bdfe-949d53cc3f5c service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:31:12.526500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-db9f67a2-137a-4501-bdfe-949d53cc3f5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:31:12.543010 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f67a2-137a-4501-bdfe-949d53cc3f5c service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:31:12.548818 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b9ef0-10fd-43ab-ac5c-4e76a6e47434 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:31:12.549458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b9ef0-10fd-43ab-ac5c-4e76a6e47434 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:31:12.549747 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b9ef0-10fd-43ab-ac5c-4e76a6e47434 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:31:12.550015 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b9ef0-10fd-43ab-ac5c-4e76a6e47434 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:31:12.551766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-326b9ef0-10fd-43ab-ac5c-4e76a6e47434 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:31:12.563336 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b9ef0-10fd-43ab-ac5c-4e76a6e47434 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:31:12.564892 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b9ef0-10fd-43ab-ac5c-4e76a6e47434 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:31:12.575308 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1376/2780] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:31:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:31:13.625353 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=40 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:31:15.547237 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa37ad4-76b1-4008-812b-7b4bd85e10af service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:31:15.560010 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa37ad4-76b1-4008-812b-7b4bd85e10af service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:31:15.566220 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffe980-1636-4150-a8c7-adc3726b4c97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:31:15.566869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffe980-1636-4150-a8c7-adc3726b4c97 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:31:15.568019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffe980-1636-4150-a8c7-adc3726b4c97 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:31:15.568019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffe980-1636-4150-a8c7-adc3726b4c97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:31:15.569550 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b8ffe980-1636-4150-a8c7-adc3726b4c97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:31:15.583858 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ffe980-1636-4150-a8c7-adc3726b4c97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:31:15.585460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ffe980-1636-4150-a8c7-adc3726b4c97 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:31:15.597158 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1405/2781] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:31:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:31:22.570906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=78 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:31:25.593295 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:31:28.188278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b9ef0-10fd-43ab-ac5c-4e76a6e47434 service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:31:28.207419 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b9ef0-10fd-43ab-ac5c-4e76a6e47434 service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:31:28.210973 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcd2c3-34e0-44a0-a815-1640cf0e9e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:31:28.211640 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcd2c3-34e0-44a0-a815-1640cf0e9e97 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:31:28.211922 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcd2c3-34e0-44a0-a815-1640cf0e9e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:31:28.212195 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcd2c3-34e0-44a0-a815-1640cf0e9e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:31:28.213967 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0adcd2c3-34e0-44a0-a815-1640cf0e9e97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:31:28.248060 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcd2c3-34e0-44a0-a815-1640cf0e9e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:31:28.249484 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcd2c3-34e0-44a0-a815-1640cf0e9e97 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:31:28.268508 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1377/2782] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:31:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:31:31.613951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffe980-1636-4150-a8c7-adc3726b4c97 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:31:31.626259 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffe980-1636-4150-a8c7-adc3726b4c97 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:31:31.631459 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f3c3a-d904-44a3-a82c-a38b61e5dd8d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:31:31.632104 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f3c3a-d904-44a3-a82c-a38b61e5dd8d service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:31:31.632388 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f3c3a-d904-44a3-a82c-a38b61e5dd8d service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:31:31.632665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f3c3a-d904-44a3-a82c-a38b61e5dd8d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:31:31.634454 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-991f3c3a-d904-44a3-a82c-a38b61e5dd8d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:31:31.646869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f3c3a-d904-44a3-a82c-a38b61e5dd8d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:31:31.648494 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f3c3a-d904-44a3-a82c-a38b61e5dd8d service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:31:31.658769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1406/2783] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:31:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:31:34.954305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcd2c3-34e0-44a0-a815-1640cf0e9e97 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:31:34.967762 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcd2c3-34e0-44a0-a815-1640cf0e9e97 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:31:34.973692 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20bf65-9e31-4eba-8136-bf84cda93984 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:31:34.974340 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20bf65-9e31-4eba-8136-bf84cda93984 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:31:34.974622 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20bf65-9e31-4eba-8136-bf84cda93984 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:31:34.974898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20bf65-9e31-4eba-8136-bf84cda93984 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:31:34.982585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2b20bf65-9e31-4eba-8136-bf84cda93984 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:31:34.995235 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20bf65-9e31-4eba-8136-bf84cda93984 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:31:34.996829 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20bf65-9e31-4eba-8136-bf84cda93984 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:31:35.007786 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1378/2784] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:31:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:31:35.344806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f3c3a-d904-44a3-a82c-a38b61e5dd8d service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:31:35.346339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-991f3c3a-d904-44a3-a82c-a38b61e5dd8d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:31:35.360871 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f3c3a-d904-44a3-a82c-a38b61e5dd8d service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:31:35.366128 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabe1f2-e7a8-494a-b009-5f8dc589ab1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:31:35.366415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabe1f2-e7a8-494a-b009-5f8dc589ab1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:31:35.366705 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabe1f2-e7a8-494a-b009-5f8dc589ab1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:31:35.367016 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabe1f2-e7a8-494a-b009-5f8dc589ab1a admin admin] PATH_INFO: `/v3/users/6f9c73ca3c364117993dac93acf5a737` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:31:35.368257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabe1f2-e7a8-494a-b009-5f8dc589ab1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f9c73ca3c364117993dac93acf5a737)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:31:35.370172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabe1f2-e7a8-494a-b009-5f8dc589ab1a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:31:35.545815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5dabe1f2-e7a8-494a-b009-5f8dc589ab1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f9c73ca3c364117993dac93acf5a737'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:35.558755 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5dabe1f2-e7a8-494a-b009-5f8dc589ab1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f9c73ca3c364117993dac93acf5a737'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:35.618316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5dabe1f2-e7a8-494a-b009-5f8dc589ab1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f9c73ca3c364117993dac93acf5a737'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:35.618930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5dabe1f2-e7a8-494a-b009-5f8dc589ab1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f9c73ca3c364117993dac93acf5a737'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:35.626660 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-5dabe1f2-e7a8-494a-b009-5f8dc589ab1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f9c73ca3c364117993dac93acf5a737'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:35.632507 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1407/2785] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:31:35 2022] DELETE /identity/v3/users/6f9c73ca3c364117993dac93acf5a737 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:31:35.866085 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20bf65-9e31-4eba-8136-bf84cda93984 service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:31:35.877620 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20bf65-9e31-4eba-8136-bf84cda93984 service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:31:35.936094 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ce6a4-ce4d-47e0-ba82-1b30008868a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:31:35.937586 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ce6a4-ce4d-47e0-ba82-1b30008868a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:31:35.938067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ce6a4-ce4d-47e0-ba82-1b30008868a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:31:35.938365 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ce6a4-ce4d-47e0-ba82-1b30008868a2 admin admin] PATH_INFO: `/v3/projects/06aac4c0b410452fa71221434149e3b7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:31:35.939586 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ce6a4-ce4d-47e0-ba82-1b30008868a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06aac4c0b410452fa71221434149e3b7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:31:35.941490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ce6a4-ce4d-47e0-ba82-1b30008868a2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:31:36.040890 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-000ce6a4-ce4d-47e0-ba82-1b30008868a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06aac4c0b410452fa71221434149e3b7', 'request_id': 'req-000ce6a4-ce4d-47e0-ba82-1b30008868a2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:36.046955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-000ce6a4-ce4d-47e0-ba82-1b30008868a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06aac4c0b410452fa71221434149e3b7', 'request_id': 'req-000ce6a4-ce4d-47e0-ba82-1b30008868a2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:36.104311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-000ce6a4-ce4d-47e0-ba82-1b30008868a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06aac4c0b410452fa71221434149e3b7', 'request_id': 'req-000ce6a4-ce4d-47e0-ba82-1b30008868a2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:36.115473 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-000ce6a4-ce4d-47e0-ba82-1b30008868a2 admin admin] The token cache is being invalidate because project 06aac4c0b410452fa71221434149e3b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:31:36.115702 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-000ce6a4-ce4d-47e0-ba82-1b30008868a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:36.116330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-000ce6a4-ce4d-47e0-ba82-1b30008868a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:36.117896 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1379/2786] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:31:35 2022] DELETE /identity/v3/projects/06aac4c0b410452fa71221434149e3b7 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:31:36.122009 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabe1f2-e7a8-494a-b009-5f8dc589ab1a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:31:36.123544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5dabe1f2-e7a8-494a-b009-5f8dc589ab1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:31:36.137565 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabe1f2-e7a8-494a-b009-5f8dc589ab1a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:31:36.194986 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be07027-325b-468d-8485-0135d931df8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:31:36.195991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be07027-325b-468d-8485-0135d931df8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:31:36.196272 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be07027-325b-468d-8485-0135d931df8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:31:36.196547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be07027-325b-468d-8485-0135d931df8c admin admin] PATH_INFO: `/v3/users/1bd375ff8eea4375836e31dee2e9b364` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:31:36.197736 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be07027-325b-468d-8485-0135d931df8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bd375ff8eea4375836e31dee2e9b364)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:31:36.199683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be07027-325b-468d-8485-0135d931df8c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:31:36.414811 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6be07027-325b-468d-8485-0135d931df8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bd375ff8eea4375836e31dee2e9b364'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:36.425694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:31:36.431416 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6be07027-325b-468d-8485-0135d931df8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bd375ff8eea4375836e31dee2e9b364'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:36.498980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6be07027-325b-468d-8485-0135d931df8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bd375ff8eea4375836e31dee2e9b364'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:36.500319 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6be07027-325b-468d-8485-0135d931df8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bd375ff8eea4375836e31dee2e9b364'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:36.507747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6be07027-325b-468d-8485-0135d931df8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bd375ff8eea4375836e31dee2e9b364'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:36.514899 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1408/2787] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:31:36 2022] DELETE /identity/v3/users/1bd375ff8eea4375836e31dee2e9b364 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:31:36.735139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ce6a4-ce4d-47e0-ba82-1b30008868a2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:31:36.738547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-000ce6a4-ce4d-47e0-ba82-1b30008868a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:31:36.749741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ce6a4-ce4d-47e0-ba82-1b30008868a2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:31:36.774997 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:31:36.807901 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ca4d4-e45d-46aa-ab7a-51e19219bad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:31:36.808770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ca4d4-e45d-46aa-ab7a-51e19219bad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:31:36.809048 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ca4d4-e45d-46aa-ab7a-51e19219bad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:31:36.809323 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ca4d4-e45d-46aa-ab7a-51e19219bad6 admin admin] PATH_INFO: `/v3/projects/d3d6a581ccf64fa6a40a8fbad19a6567` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:31:36.810479 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ca4d4-e45d-46aa-ab7a-51e19219bad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3d6a581ccf64fa6a40a8fbad19a6567)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:31:36.812491 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ca4d4-e45d-46aa-ab7a-51e19219bad6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:31:36.904147 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-940ca4d4-e45d-46aa-ab7a-51e19219bad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3d6a581ccf64fa6a40a8fbad19a6567', 'request_id': 'req-940ca4d4-e45d-46aa-ab7a-51e19219bad6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:36.910861 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-940ca4d4-e45d-46aa-ab7a-51e19219bad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3d6a581ccf64fa6a40a8fbad19a6567', 'request_id': 'req-940ca4d4-e45d-46aa-ab7a-51e19219bad6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:36.984935 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-940ca4d4-e45d-46aa-ab7a-51e19219bad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3d6a581ccf64fa6a40a8fbad19a6567', 'request_id': 'req-940ca4d4-e45d-46aa-ab7a-51e19219bad6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:36.996437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-940ca4d4-e45d-46aa-ab7a-51e19219bad6 admin admin] The token cache is being invalidate because project d3d6a581ccf64fa6a40a8fbad19a6567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:31:36.996727 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-940ca4d4-e45d-46aa-ab7a-51e19219bad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:36.997352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-940ca4d4-e45d-46aa-ab7a-51e19219bad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:31:36.998889 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1380/2788] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:31:36 2022] DELETE /identity/v3/projects/d3d6a581ccf64fa6a40a8fbad19a6567 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:31:46.641325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:31:46.993343 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:32:03.299342 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a171442-3f41-4d7a-8717-2480fc592cb2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:32:03.299930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a171442-3f41-4d7a-8717-2480fc592cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:32:03.299930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a171442-3f41-4d7a-8717-2480fc592cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:32:03.318185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4a171442-3f41-4d7a-8717-2480fc592cb2 None None] MFA Rules not processed for user `920e7c03e9b94550880b452d84f07128`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:32:03.319460 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4a171442-3f41-4d7a-8717-2480fc592cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:32:03.373525 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1409/2789] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:32:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:32:09.198650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ca4d4-e45d-46aa-ab7a-51e19219bad6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:32:09.200084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-940ca4d4-e45d-46aa-ab7a-51e19219bad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:32:09.291237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ca4d4-e45d-46aa-ab7a-51e19219bad6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:32:09.296283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528637f-20f0-4e78-befc-752fb88ae539 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:32:09.296931 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528637f-20f0-4e78-befc-752fb88ae539 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:32:09.297219 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528637f-20f0-4e78-befc-752fb88ae539 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:32:09.297494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528637f-20f0-4e78-befc-752fb88ae539 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:32:09.299271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f528637f-20f0-4e78-befc-752fb88ae539 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:32:09.310581 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528637f-20f0-4e78-befc-752fb88ae539 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:32:09.312162 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528637f-20f0-4e78-befc-752fb88ae539 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:32:09.322215 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1381/2790] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:32:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:32:13.366962 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=25 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:32:19.317836 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=41 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:33:08.156572 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5432f3-65f6-4db5-868d-c672ebc9385a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:08.158547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5432f3-65f6-4db5-868d-c672ebc9385a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:08.158547 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5432f3-65f6-4db5-868d-c672ebc9385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:08.175684 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3b5432f3-65f6-4db5-868d-c672ebc9385a None None] MFA Rules not processed for user `920e7c03e9b94550880b452d84f07128`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:33:08.176924 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3b5432f3-65f6-4db5-868d-c672ebc9385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:08.183835 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1410/2791] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:33:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:33:18.170479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:33:59.533066 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528637f-20f0-4e78-befc-752fb88ae539 service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:59.534714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f528637f-20f0-4e78-befc-752fb88ae539 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:59.620019 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528637f-20f0-4e78-befc-752fb88ae539 service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:59.623742 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda649f-8e04-49c4-a80d-07d13a98995e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:59.624985 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda649f-8e04-49c4-a80d-07d13a98995e service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:59.625271 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda649f-8e04-49c4-a80d-07d13a98995e service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:59.625552 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda649f-8e04-49c4-a80d-07d13a98995e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:59.627425 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2cda649f-8e04-49c4-a80d-07d13a98995e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:59.716627 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda649f-8e04-49c4-a80d-07d13a98995e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:59.718777 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda649f-8e04-49c4-a80d-07d13a98995e service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:59.738602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1382/2792] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:33:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:34:08.009009 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5432f3-65f6-4db5-868d-c672ebc9385a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:08.018965 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5432f3-65f6-4db5-868d-c672ebc9385a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:08.022813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13637c-06cd-4659-b322-d80924a6154e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:08.023484 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13637c-06cd-4659-b322-d80924a6154e service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:08.026426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13637c-06cd-4659-b322-d80924a6154e service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:08.026426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13637c-06cd-4659-b322-d80924a6154e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:08.026426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6a13637c-06cd-4659-b322-d80924a6154e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:08.039969 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13637c-06cd-4659-b322-d80924a6154e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:08.041446 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13637c-06cd-4659-b322-d80924a6154e service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:08.053297 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1411/2793] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:09.735983 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=75 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:16.336770 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebf516-4e4a-46ff-840a-10586946c292 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:16.337364 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebf516-4e4a-46ff-840a-10586946c292 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:16.337464 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebf516-4e4a-46ff-840a-10586946c292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:16.355676 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-60ebf516-4e4a-46ff-840a-10586946c292 None None] MFA Rules not processed for user `920e7c03e9b94550880b452d84f07128`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:16.356804 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-60ebf516-4e4a-46ff-840a-10586946c292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:16.364702 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1383/2794] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:34:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:18.048349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:19.706090 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13637c-06cd-4659-b322-d80924a6154e service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:19.707554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6a13637c-06cd-4659-b322-d80924a6154e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:19.777302 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13637c-06cd-4659-b322-d80924a6154e service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:19.780876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618d345-24ea-4cb7-aa75-00f107c83492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:19.781512 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618d345-24ea-4cb7-aa75-00f107c83492 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:19.781793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618d345-24ea-4cb7-aa75-00f107c83492 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:19.782064 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618d345-24ea-4cb7-aa75-00f107c83492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:19.784558 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3618d345-24ea-4cb7-aa75-00f107c83492 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:19.794376 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618d345-24ea-4cb7-aa75-00f107c83492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:19.795867 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618d345-24ea-4cb7-aa75-00f107c83492 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:19.804514 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1412/2795] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:26.350238 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:29.801366 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=31 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:29.918805 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebf516-4e4a-46ff-840a-10586946c292 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:29.920400 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-60ebf516-4e4a-46ff-840a-10586946c292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:29.987950 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebf516-4e4a-46ff-840a-10586946c292 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:29.992556 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9414b-b772-4092-bd39-3719d832474f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:29.993439 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9414b-b772-4092-bd39-3719d832474f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:29.993718 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9414b-b772-4092-bd39-3719d832474f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:29.993990 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9414b-b772-4092-bd39-3719d832474f admin admin] PATH_INFO: `/v3/users/920e7c03e9b94550880b452d84f07128` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:29.995341 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9414b-b772-4092-bd39-3719d832474f admin admin] RBAC: Authorizing `identity:delete_user(user_id=920e7c03e9b94550880b452d84f07128)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:29.997310 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9414b-b772-4092-bd39-3719d832474f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:30.189326 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4cf9414b-b772-4092-bd39-3719d832474f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920e7c03e9b94550880b452d84f07128'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:30.201593 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4cf9414b-b772-4092-bd39-3719d832474f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920e7c03e9b94550880b452d84f07128'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:30.202272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4cf9414b-b772-4092-bd39-3719d832474f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920e7c03e9b94550880b452d84f07128'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:30.209079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4cf9414b-b772-4092-bd39-3719d832474f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920e7c03e9b94550880b452d84f07128'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:30.210283 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4cf9414b-b772-4092-bd39-3719d832474f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920e7c03e9b94550880b452d84f07128'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:30.277000 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1384/2796] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:35:29 2022] DELETE /identity/v3/users/920e7c03e9b94550880b452d84f07128 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:30.582142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618d345-24ea-4cb7-aa75-00f107c83492 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:30.595648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618d345-24ea-4cb7-aa75-00f107c83492 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:30.653000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c83e4-4239-455e-81d4-231e23686b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:30.653896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c83e4-4239-455e-81d4-231e23686b8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:30.654268 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c83e4-4239-455e-81d4-231e23686b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:30.654486 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c83e4-4239-455e-81d4-231e23686b8f admin admin] PATH_INFO: `/v3/projects/9c0e7447465d430d9352d9e9b7ea012c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:30.655690 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c83e4-4239-455e-81d4-231e23686b8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c0e7447465d430d9352d9e9b7ea012c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:30.657586 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c83e4-4239-455e-81d4-231e23686b8f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:30.786666 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-710c83e4-4239-455e-81d4-231e23686b8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c0e7447465d430d9352d9e9b7ea012c', 'request_id': 'req-710c83e4-4239-455e-81d4-231e23686b8f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:30.793437 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-710c83e4-4239-455e-81d4-231e23686b8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c0e7447465d430d9352d9e9b7ea012c', 'request_id': 'req-710c83e4-4239-455e-81d4-231e23686b8f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:30.854464 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-710c83e4-4239-455e-81d4-231e23686b8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c0e7447465d430d9352d9e9b7ea012c', 'request_id': 'req-710c83e4-4239-455e-81d4-231e23686b8f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:30.865637 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-710c83e4-4239-455e-81d4-231e23686b8f admin admin] The token cache is being invalidate because project 9c0e7447465d430d9352d9e9b7ea012c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:35:30.866576 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-710c83e4-4239-455e-81d4-231e23686b8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:30.867439 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-710c83e4-4239-455e-81d4-231e23686b8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:30.869106 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1413/2797] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:35:30 2022] DELETE /identity/v3/projects/9c0e7447465d430d9352d9e9b7ea012c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:30.874089 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fd132-6188-4654-bb5f-abea08615259 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:30.874385 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fd132-6188-4654-bb5f-abea08615259 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:30.874673 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fd132-6188-4654-bb5f-abea08615259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:30.930991 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6c2fd132-6188-4654-bb5f-abea08615259 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:30.932306 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6c2fd132-6188-4654-bb5f-abea08615259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:30.974665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:30.989763 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1385/2798] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:30.999604 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a55dd-97fb-4b07-a5e5-0dcdb577e0c0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:30.999891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a55dd-97fb-4b07-a5e5-0dcdb577e0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:31.000167 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a55dd-97fb-4b07-a5e5-0dcdb577e0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:31.018498 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b45a55dd-97fb-4b07-a5e5-0dcdb577e0c0 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:31.020014 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b45a55dd-97fb-4b07-a5e5-0dcdb577e0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:31.028261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1414/2799] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:31.032476 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2fd132-6188-4654-bb5f-abea08615259 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:31.042989 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2fd132-6188-4654-bb5f-abea08615259 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:31.048030 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2fc22-5544-4c6e-9f52-9fdb1982a87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:31.048691 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2fc22-5544-4c6e-9f52-9fdb1982a87b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:31.048974 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2fc22-5544-4c6e-9f52-9fdb1982a87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:31.049251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2fc22-5544-4c6e-9f52-9fdb1982a87b admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:31.049970 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2fc22-5544-4c6e-9f52-9fdb1982a87b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:35:31.050277 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2fc22-5544-4c6e-9f52-9fdb1982a87b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:31.051910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2fc22-5544-4c6e-9f52-9fdb1982a87b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:31.063867 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1386/2800] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:35:31.068129 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a55dd-97fb-4b07-a5e5-0dcdb577e0c0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:31.077402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a55dd-97fb-4b07-a5e5-0dcdb577e0c0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:31.082082 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93c975-0f6e-4a62-82f3-ef4ba25d7b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:31.082722 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93c975-0f6e-4a62-82f3-ef4ba25d7b15 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:31.083002 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93c975-0f6e-4a62-82f3-ef4ba25d7b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:31.083313 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93c975-0f6e-4a62-82f3-ef4ba25d7b15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:31.083991 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93c975-0f6e-4a62-82f3-ef4ba25d7b15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:31.086477 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93c975-0f6e-4a62-82f3-ef4ba25d7b15 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:31.181098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1415/2801] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:35:31 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:35:31.185190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2fc22-5544-4c6e-9f52-9fdb1982a87b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:31.195416 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2fc22-5544-4c6e-9f52-9fdb1982a87b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:31.251541 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c5d19-0d83-43a6-958a-db0a127b4304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:31.252204 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c5d19-0d83-43a6-958a-db0a127b4304 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:31.252489 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c5d19-0d83-43a6-958a-db0a127b4304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:31.252767 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c5d19-0d83-43a6-958a-db0a127b4304 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:31.253463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c5d19-0d83-43a6-958a-db0a127b4304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:31.255402 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c5d19-0d83-43a6-958a-db0a127b4304 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:31.322529 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1387/2802] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:35:31 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:35:31.326809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93c975-0f6e-4a62-82f3-ef4ba25d7b15 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:31.338426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93c975-0f6e-4a62-82f3-ef4ba25d7b15 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:31.343069 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b2a24-e78d-4036-a118-2a16aa28e3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:31.343748 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b2a24-e78d-4036-a118-2a16aa28e3db admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:31.344030 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b2a24-e78d-4036-a118-2a16aa28e3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:31.344305 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b2a24-e78d-4036-a118-2a16aa28e3db admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:31.345019 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b2a24-e78d-4036-a118-2a16aa28e3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:31.346597 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b2a24-e78d-4036-a118-2a16aa28e3db admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:31.356891 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1416/2803] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:35:31.360744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c5d19-0d83-43a6-958a-db0a127b4304 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:31.370720 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c5d19-0d83-43a6-958a-db0a127b4304 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:31.375498 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad365e1-28ba-4f2b-abb2-31c9face2e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:31.376276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad365e1-28ba-4f2b-abb2-31c9face2e30 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:31.376578 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad365e1-28ba-4f2b-abb2-31c9face2e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:31.376858 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad365e1-28ba-4f2b-abb2-31c9face2e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:31.377581 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad365e1-28ba-4f2b-abb2-31c9face2e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:31.379164 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad365e1-28ba-4f2b-abb2-31c9face2e30 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:31.389496 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1388/2804] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:35:31.393411 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b2a24-e78d-4036-a118-2a16aa28e3db admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:31.404749 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b2a24-e78d-4036-a118-2a16aa28e3db admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:31.409612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ef1ad-ce3b-4c9d-b23f-4b38ba85fc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:31.410494 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ef1ad-ce3b-4c9d-b23f-4b38ba85fc50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:31.410777 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ef1ad-ce3b-4c9d-b23f-4b38ba85fc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:31.411056 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ef1ad-ce3b-4c9d-b23f-4b38ba85fc50 admin admin] PATH_INFO: `/v3/projects/dd884e57908d4b01acf9d1254c68fc70/users/594b74104d494400b537afd2b7d156d8/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:31.427538 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ef1ad-ce3b-4c9d-b23f-4b38ba85fc50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd884e57908d4b01acf9d1254c68fc70, user_id=594b74104d494400b537afd2b7d156d8, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:31.430559 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ef1ad-ce3b-4c9d-b23f-4b38ba85fc50 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:31.477809 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1417/2805] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:35:31 2022] PUT /identity/v3/projects/dd884e57908d4b01acf9d1254c68fc70/users/594b74104d494400b537afd2b7d156d8/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:31.487891 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e20a8-7b47-4213-ad33-f51a898a5720 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:31.488209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e20a8-7b47-4213-ad33-f51a898a5720 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:31.488492 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e20a8-7b47-4213-ad33-f51a898a5720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:31.514827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d02e20a8-7b47-4213-ad33-f51a898a5720 None None] MFA Rules not processed for user `594b74104d494400b537afd2b7d156d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:31.516063 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d02e20a8-7b47-4213-ad33-f51a898a5720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:31.578746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1389/2806] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:35:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:31.583172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ef1ad-ce3b-4c9d-b23f-4b38ba85fc50 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:31.602734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:31.643638 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ef1ad-ce3b-4c9d-b23f-4b38ba85fc50 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:31.649265 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1c1ac-d3d6-4120-84f5-e3202c8164f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:31.649916 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1c1ac-d3d6-4120-84f5-e3202c8164f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:31.650200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1c1ac-d3d6-4120-84f5-e3202c8164f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:31.650505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1c1ac-d3d6-4120-84f5-e3202c8164f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:31.651209 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1c1ac-d3d6-4120-84f5-e3202c8164f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:31.653103 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1c1ac-d3d6-4120-84f5-e3202c8164f3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:31.741224 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1418/2807] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:35:31 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:35:31.745311 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e20a8-7b47-4213-ad33-f51a898a5720 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:31.755968 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e20a8-7b47-4213-ad33-f51a898a5720 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:31.798167 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:31.812917 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74f871-ed7d-4329-908b-150260a4f5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:31.813585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74f871-ed7d-4329-908b-150260a4f5bd admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:31.813871 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74f871-ed7d-4329-908b-150260a4f5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:31.814151 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74f871-ed7d-4329-908b-150260a4f5bd admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:31.814848 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74f871-ed7d-4329-908b-150260a4f5bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:31.816784 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74f871-ed7d-4329-908b-150260a4f5bd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:31.889369 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1390/2808] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:35:31 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:35:31.893428 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1c1ac-d3d6-4120-84f5-e3202c8164f3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:31.904098 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1c1ac-d3d6-4120-84f5-e3202c8164f3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:31.908743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bfd2e-91d5-4507-ab62-d50e40014318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:31.909396 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bfd2e-91d5-4507-ab62-d50e40014318 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:31.909689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bfd2e-91d5-4507-ab62-d50e40014318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:31.909968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bfd2e-91d5-4507-ab62-d50e40014318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:31.910694 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bfd2e-91d5-4507-ab62-d50e40014318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:31.912318 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bfd2e-91d5-4507-ab62-d50e40014318 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:31.922770 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1419/2809] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:35:31.927048 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74f871-ed7d-4329-908b-150260a4f5bd admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:31.938125 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74f871-ed7d-4329-908b-150260a4f5bd admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:31.942812 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62b9d2-b56e-44c8-8197-f3edc504f72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:31.943734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62b9d2-b56e-44c8-8197-f3edc504f72a admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:31.944067 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62b9d2-b56e-44c8-8197-f3edc504f72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:31.945057 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62b9d2-b56e-44c8-8197-f3edc504f72a admin admin] PATH_INFO: `/v3/projects/7c2f195d754641dab423bf2887d163fa/users/72b24fb152054f8fbe9e73a1b564565f/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:31.959883 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62b9d2-b56e-44c8-8197-f3edc504f72a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c2f195d754641dab423bf2887d163fa, user_id=72b24fb152054f8fbe9e73a1b564565f, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:31.962913 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62b9d2-b56e-44c8-8197-f3edc504f72a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:32.031734 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1391/2810] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:35:31 2022] PUT /identity/v3/projects/7c2f195d754641dab423bf2887d163fa/users/72b24fb152054f8fbe9e73a1b564565f/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:32.035596 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bfd2e-91d5-4507-ab62-d50e40014318 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:32.045664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bfd2e-91d5-4507-ab62-d50e40014318 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:32.100895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458b198-c584-42cd-8f17-05dcd81166dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:32.101567 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458b198-c584-42cd-8f17-05dcd81166dc admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:32.101848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458b198-c584-42cd-8f17-05dcd81166dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:32.102123 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458b198-c584-42cd-8f17-05dcd81166dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:32.102843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458b198-c584-42cd-8f17-05dcd81166dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:32.104501 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458b198-c584-42cd-8f17-05dcd81166dc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:32.114225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1420/2811] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:35:32.118883 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62b9d2-b56e-44c8-8197-f3edc504f72a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:32.129054 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62b9d2-b56e-44c8-8197-f3edc504f72a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:32.134766 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1befe5f-ab57-487b-a8e2-de3ad7c364b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:32.135650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1befe5f-ab57-487b-a8e2-de3ad7c364b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:32.135936 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1befe5f-ab57-487b-a8e2-de3ad7c364b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:32.136213 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1befe5f-ab57-487b-a8e2-de3ad7c364b5 admin admin] PATH_INFO: `/v3/projects/7c2f195d754641dab423bf2887d163fa/users/72b24fb152054f8fbe9e73a1b564565f/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:32.138159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1befe5f-ab57-487b-a8e2-de3ad7c364b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c2f195d754641dab423bf2887d163fa, user_id=72b24fb152054f8fbe9e73a1b564565f, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:32.140799 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1befe5f-ab57-487b-a8e2-de3ad7c364b5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:32.151449 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1392/2812] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:35:32 2022] PUT /identity/v3/projects/7c2f195d754641dab423bf2887d163fa/users/72b24fb152054f8fbe9e73a1b564565f/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:32.162494 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a131b2-614a-4114-8be5-372951d8226d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:32.162896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a131b2-614a-4114-8be5-372951d8226d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:32.163236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a131b2-614a-4114-8be5-372951d8226d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:32.189584 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b0a131b2-614a-4114-8be5-372951d8226d None None] MFA Rules not processed for user `72b24fb152054f8fbe9e73a1b564565f`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:32.190775 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b0a131b2-614a-4114-8be5-372951d8226d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:32.260277 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1421/2813] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:35:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:32.271790 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1befe5f-ab57-487b-a8e2-de3ad7c364b5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:32.273589 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d1befe5f-ab57-487b-a8e2-de3ad7c364b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:32.330547 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1befe5f-ab57-487b-a8e2-de3ad7c364b5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:32.334065 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8d3aa-6087-453a-9cf1-8527ec238a66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:32.334733 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8d3aa-6087-453a-9cf1-8527ec238a66 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:32.335009 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8d3aa-6087-453a-9cf1-8527ec238a66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:32.335316 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8d3aa-6087-453a-9cf1-8527ec238a66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:32.344108 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8d3aa-6087-453a-9cf1-8527ec238a66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:32.345628 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8d3aa-6087-453a-9cf1-8527ec238a66 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:32.355459 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1393/2814] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:35:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:35:33.237085 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a131b2-614a-4114-8be5-372951d8226d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:33.250403 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:33.301883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a131b2-614a-4114-8be5-372951d8226d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:33.307276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382478be-294e-4b26-b2e0-1dbdb4b798d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:33.308153 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382478be-294e-4b26-b2e0-1dbdb4b798d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:33.308435 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382478be-294e-4b26-b2e0-1dbdb4b798d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:33.308712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382478be-294e-4b26-b2e0-1dbdb4b798d3 admin admin] PATH_INFO: `/v3/users/594b74104d494400b537afd2b7d156d8` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:33.309883 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382478be-294e-4b26-b2e0-1dbdb4b798d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=594b74104d494400b537afd2b7d156d8)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:33.312183 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382478be-294e-4b26-b2e0-1dbdb4b798d3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:33.474144 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-382478be-294e-4b26-b2e0-1dbdb4b798d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '594b74104d494400b537afd2b7d156d8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:33.486431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-382478be-294e-4b26-b2e0-1dbdb4b798d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '594b74104d494400b537afd2b7d156d8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:33.545538 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-382478be-294e-4b26-b2e0-1dbdb4b798d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '594b74104d494400b537afd2b7d156d8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:33.546261 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-382478be-294e-4b26-b2e0-1dbdb4b798d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '594b74104d494400b537afd2b7d156d8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:33.553823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-382478be-294e-4b26-b2e0-1dbdb4b798d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '594b74104d494400b537afd2b7d156d8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:33.559741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1422/2815] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:35:33 2022] DELETE /identity/v3/users/594b74104d494400b537afd2b7d156d8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:33.564321 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8832b-b95f-4b1c-944e-3f6b6e643870 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:33.564789 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8832b-b95f-4b1c-944e-3f6b6e643870 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:33.564904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8832b-b95f-4b1c-944e-3f6b6e643870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:33.582417 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d6f8832b-b95f-4b1c-944e-3f6b6e643870 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:33.583679 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d6f8832b-b95f-4b1c-944e-3f6b6e643870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:33.641507 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1394/2816] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:33.651989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382478be-294e-4b26-b2e0-1dbdb4b798d3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:33.653404 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-382478be-294e-4b26-b2e0-1dbdb4b798d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:33.709177 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382478be-294e-4b26-b2e0-1dbdb4b798d3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:33.713496 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ddf6b-760a-4cf1-a7a7-69a439040688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:33.714131 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ddf6b-760a-4cf1-a7a7-69a439040688 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:33.714412 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ddf6b-760a-4cf1-a7a7-69a439040688 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:33.714686 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ddf6b-760a-4cf1-a7a7-69a439040688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:33.723473 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ddf6b-760a-4cf1-a7a7-69a439040688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:33.724949 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ddf6b-760a-4cf1-a7a7-69a439040688 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:33.734931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1423/2817] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:35:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:35:34.170701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8832b-b95f-4b1c-944e-3f6b6e643870 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:34.173297 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d6f8832b-b95f-4b1c-944e-3f6b6e643870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:34.184932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8832b-b95f-4b1c-944e-3f6b6e643870 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:34.189538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ac5b9-ee80-4f24-9811-94cc5fe22675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:34.190410 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ac5b9-ee80-4f24-9811-94cc5fe22675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:34.190695 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ac5b9-ee80-4f24-9811-94cc5fe22675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:34.190971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ac5b9-ee80-4f24-9811-94cc5fe22675 admin admin] PATH_INFO: `/v3/projects/dd884e57908d4b01acf9d1254c68fc70` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:34.192182 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ac5b9-ee80-4f24-9811-94cc5fe22675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd884e57908d4b01acf9d1254c68fc70)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:34.194828 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ac5b9-ee80-4f24-9811-94cc5fe22675 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:34.274001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:34.283301 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-756ac5b9-ee80-4f24-9811-94cc5fe22675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd884e57908d4b01acf9d1254c68fc70', 'request_id': 'req-756ac5b9-ee80-4f24-9811-94cc5fe22675'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:34.289372 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-756ac5b9-ee80-4f24-9811-94cc5fe22675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd884e57908d4b01acf9d1254c68fc70', 'request_id': 'req-756ac5b9-ee80-4f24-9811-94cc5fe22675'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:34.356780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-756ac5b9-ee80-4f24-9811-94cc5fe22675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd884e57908d4b01acf9d1254c68fc70', 'request_id': 'req-756ac5b9-ee80-4f24-9811-94cc5fe22675'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:34.367838 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-756ac5b9-ee80-4f24-9811-94cc5fe22675 admin admin] The token cache is being invalidate because project dd884e57908d4b01acf9d1254c68fc70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:35:34.368128 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-756ac5b9-ee80-4f24-9811-94cc5fe22675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:34.368762 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-756ac5b9-ee80-4f24-9811-94cc5fe22675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:34.370304 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1395/2818] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:35:34 2022] DELETE /identity/v3/projects/dd884e57908d4b01acf9d1254c68fc70 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:34.374395 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ddf6b-760a-4cf1-a7a7-69a439040688 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:34.384583 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ddf6b-760a-4cf1-a7a7-69a439040688 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:34.417112 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:34.440631 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbefe9-dd26-4a79-9265-b537a58bea2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:34.441502 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbefe9-dd26-4a79-9265-b537a58bea2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:34.441812 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbefe9-dd26-4a79-9265-b537a58bea2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:34.442111 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbefe9-dd26-4a79-9265-b537a58bea2f admin admin] PATH_INFO: `/v3/users/72b24fb152054f8fbe9e73a1b564565f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:34.443319 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbefe9-dd26-4a79-9265-b537a58bea2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=72b24fb152054f8fbe9e73a1b564565f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:34.445969 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbefe9-dd26-4a79-9265-b537a58bea2f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:34.587582 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-facbefe9-dd26-4a79-9265-b537a58bea2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72b24fb152054f8fbe9e73a1b564565f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:34.599349 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-facbefe9-dd26-4a79-9265-b537a58bea2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72b24fb152054f8fbe9e73a1b564565f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:34.682955 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-facbefe9-dd26-4a79-9265-b537a58bea2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72b24fb152054f8fbe9e73a1b564565f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:34.683598 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-facbefe9-dd26-4a79-9265-b537a58bea2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72b24fb152054f8fbe9e73a1b564565f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:34.690969 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-facbefe9-dd26-4a79-9265-b537a58bea2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72b24fb152054f8fbe9e73a1b564565f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:34.697335 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1424/2819] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:35:34 2022] DELETE /identity/v3/users/72b24fb152054f8fbe9e73a1b564565f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:35.089987 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ac5b9-ee80-4f24-9811-94cc5fe22675 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:35.091661 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-756ac5b9-ee80-4f24-9811-94cc5fe22675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:35.107735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ac5b9-ee80-4f24-9811-94cc5fe22675 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:35.167517 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8651de1-2a3a-4d19-a9e0-d48a2dc3ff83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:35.168385 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8651de1-2a3a-4d19-a9e0-d48a2dc3ff83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:35.168668 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8651de1-2a3a-4d19-a9e0-d48a2dc3ff83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:35.168944 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8651de1-2a3a-4d19-a9e0-d48a2dc3ff83 admin admin] PATH_INFO: `/v3/projects/7c2f195d754641dab423bf2887d163fa` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:35.170106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8651de1-2a3a-4d19-a9e0-d48a2dc3ff83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c2f195d754641dab423bf2887d163fa)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:35.172036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8651de1-2a3a-4d19-a9e0-d48a2dc3ff83 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:35.277701 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a8651de1-2a3a-4d19-a9e0-d48a2dc3ff83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c2f195d754641dab423bf2887d163fa', 'request_id': 'req-a8651de1-2a3a-4d19-a9e0-d48a2dc3ff83'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:35.283785 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a8651de1-2a3a-4d19-a9e0-d48a2dc3ff83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c2f195d754641dab423bf2887d163fa', 'request_id': 'req-a8651de1-2a3a-4d19-a9e0-d48a2dc3ff83'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:35.322837 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a8651de1-2a3a-4d19-a9e0-d48a2dc3ff83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c2f195d754641dab423bf2887d163fa', 'request_id': 'req-a8651de1-2a3a-4d19-a9e0-d48a2dc3ff83'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:35.333086 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a8651de1-2a3a-4d19-a9e0-d48a2dc3ff83 admin admin] The token cache is being invalidate because project 7c2f195d754641dab423bf2887d163fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:35:35.335305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a8651de1-2a3a-4d19-a9e0-d48a2dc3ff83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:35.335305 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a8651de1-2a3a-4d19-a9e0-d48a2dc3ff83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:35.335571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1396/2820] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:35:35 2022] DELETE /identity/v3/projects/7c2f195d754641dab423bf2887d163fa => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:35.341039 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856e9b4-29ec-4ca5-b764-5264e153660e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:35.341331 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856e9b4-29ec-4ca5-b764-5264e153660e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:35.341615 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856e9b4-29ec-4ca5-b764-5264e153660e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:35.360519 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7856e9b4-29ec-4ca5-b764-5264e153660e None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:35.361663 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7856e9b4-29ec-4ca5-b764-5264e153660e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:35.421139 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1425/2821] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:35.429756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ff121-3020-4ecf-886c-79eda1e85c14 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:35.430049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ff121-3020-4ecf-886c-79eda1e85c14 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:35.430335 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ff121-3020-4ecf-886c-79eda1e85c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:35.448883 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-856ff121-3020-4ecf-886c-79eda1e85c14 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:35.450084 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-856ff121-3020-4ecf-886c-79eda1e85c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:35.457943 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1397/2822] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:35.462113 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856e9b4-29ec-4ca5-b764-5264e153660e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:35.471673 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856e9b4-29ec-4ca5-b764-5264e153660e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:35.476766 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579da770-d942-4ca7-9c62-ddf210267da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:35.477418 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579da770-d942-4ca7-9c62-ddf210267da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:35.477698 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579da770-d942-4ca7-9c62-ddf210267da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:35.477974 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579da770-d942-4ca7-9c62-ddf210267da2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:35.478697 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579da770-d942-4ca7-9c62-ddf210267da2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:35:35.479002 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579da770-d942-4ca7-9c62-ddf210267da2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:35.480620 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579da770-d942-4ca7-9c62-ddf210267da2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:35.492645 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1426/2823] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:35:35.496898 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ff121-3020-4ecf-886c-79eda1e85c14 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:35.506122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ff121-3020-4ecf-886c-79eda1e85c14 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:35.510959 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02e92a-db15-4f3e-b668-0923008bd2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:35.511634 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02e92a-db15-4f3e-b668-0923008bd2b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:35.511919 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02e92a-db15-4f3e-b668-0923008bd2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:35.512196 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02e92a-db15-4f3e-b668-0923008bd2b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:35.512882 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b02e92a-db15-4f3e-b668-0923008bd2b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:35.514765 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b02e92a-db15-4f3e-b668-0923008bd2b7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:35.604615 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1398/2824] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:35:35 2022] POST /identity/v3/projects => generated 357 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:35:35.608813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579da770-d942-4ca7-9c62-ddf210267da2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:35.618937 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579da770-d942-4ca7-9c62-ddf210267da2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:35.658752 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:35.674996 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d434e4-f375-4a51-aa89-7b4e64c6239c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:35.675761 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d434e4-f375-4a51-aa89-7b4e64c6239c admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:35.676051 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d434e4-f375-4a51-aa89-7b4e64c6239c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:35.676329 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d434e4-f375-4a51-aa89-7b4e64c6239c admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:35.677020 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d434e4-f375-4a51-aa89-7b4e64c6239c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:35.678990 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d434e4-f375-4a51-aa89-7b4e64c6239c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:35.750903 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1427/2825] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:35:35 2022] POST /identity/v3/users => generated 279 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:35:35.754864 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02e92a-db15-4f3e-b668-0923008bd2b7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:35.766133 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02e92a-db15-4f3e-b668-0923008bd2b7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:35.771064 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63eef05-8c60-4aad-83da-03cc5795860b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:35.771758 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63eef05-8c60-4aad-83da-03cc5795860b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:35.772643 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63eef05-8c60-4aad-83da-03cc5795860b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:35.772976 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63eef05-8c60-4aad-83da-03cc5795860b admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:35.773700 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63eef05-8c60-4aad-83da-03cc5795860b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:35.775333 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63eef05-8c60-4aad-83da-03cc5795860b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:35.786602 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1399/2826] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:35:35.791808 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d434e4-f375-4a51-aa89-7b4e64c6239c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:35.801740 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d434e4-f375-4a51-aa89-7b4e64c6239c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:35.806415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02e8ee-7def-49d7-b272-66e05b49e7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:35.811940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02e8ee-7def-49d7-b272-66e05b49e7b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:35.811940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02e8ee-7def-49d7-b272-66e05b49e7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:35.811940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02e8ee-7def-49d7-b272-66e05b49e7b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:35.811940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02e8ee-7def-49d7-b272-66e05b49e7b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:35.811940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02e8ee-7def-49d7-b272-66e05b49e7b8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:35.820037 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1428/2827] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:35:35.824219 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63eef05-8c60-4aad-83da-03cc5795860b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:35.834041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63eef05-8c60-4aad-83da-03cc5795860b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:35.839159 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156098eb-5791-48b0-9fab-50a69c042644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:35.840658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156098eb-5791-48b0-9fab-50a69c042644 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:35.840658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156098eb-5791-48b0-9fab-50a69c042644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:35.840658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156098eb-5791-48b0-9fab-50a69c042644 admin admin] PATH_INFO: `/v3/projects/e482a3fd2ced474d89932ae845d7aa9c/users/caec2c1298ab45d3a244112d8775b371/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:35.852357 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:35.856275 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156098eb-5791-48b0-9fab-50a69c042644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e482a3fd2ced474d89932ae845d7aa9c, user_id=caec2c1298ab45d3a244112d8775b371, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:35.858975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156098eb-5791-48b0-9fab-50a69c042644 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:35.905582 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1400/2828] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:35:35 2022] PUT /identity/v3/projects/e482a3fd2ced474d89932ae845d7aa9c/users/caec2c1298ab45d3a244112d8775b371/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:35.916853 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db1481-f6f0-4186-a360-8ff0d101845f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:35.917142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db1481-f6f0-4186-a360-8ff0d101845f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:35.917422 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db1481-f6f0-4186-a360-8ff0d101845f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:35.944315 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a9db1481-f6f0-4186-a360-8ff0d101845f None None] MFA Rules not processed for user `caec2c1298ab45d3a244112d8775b371`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:35.945620 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a9db1481-f6f0-4186-a360-8ff0d101845f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:36.010880 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1429/2829] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:35:35 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:36.015437 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156098eb-5791-48b0-9fab-50a69c042644 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:36.076562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156098eb-5791-48b0-9fab-50a69c042644 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:36.081280 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0683dc-334c-4727-9caa-e0e74a267ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:36.084190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0683dc-334c-4727-9caa-e0e74a267ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:36.084190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0683dc-334c-4727-9caa-e0e74a267ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:36.084190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0683dc-334c-4727-9caa-e0e74a267ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:36.084190 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0683dc-334c-4727-9caa-e0e74a267ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:36.085939 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0683dc-334c-4727-9caa-e0e74a267ec9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:36.164029 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1401/2830] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:35:36 2022] POST /identity/v3/projects => generated 357 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:35:36.168243 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db1481-f6f0-4186-a360-8ff0d101845f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:36.179339 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db1481-f6f0-4186-a360-8ff0d101845f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:36.234557 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acfa1d-97bc-42d2-9d74-5f61cdb8e326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:36.235244 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acfa1d-97bc-42d2-9d74-5f61cdb8e326 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:36.235535 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acfa1d-97bc-42d2-9d74-5f61cdb8e326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:36.235813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acfa1d-97bc-42d2-9d74-5f61cdb8e326 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:36.236560 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acfa1d-97bc-42d2-9d74-5f61cdb8e326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:36.238424 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acfa1d-97bc-42d2-9d74-5f61cdb8e326 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:36.300703 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1430/2831] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:35:36 2022] POST /identity/v3/users => generated 285 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:35:36.304258 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0683dc-334c-4727-9caa-e0e74a267ec9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:36.314494 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0683dc-334c-4727-9caa-e0e74a267ec9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:36.319706 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c74085-74e4-46b3-984b-24b0d490f001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:36.320366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c74085-74e4-46b3-984b-24b0d490f001 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:36.320685 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c74085-74e4-46b3-984b-24b0d490f001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:36.320926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c74085-74e4-46b3-984b-24b0d490f001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:36.321636 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c74085-74e4-46b3-984b-24b0d490f001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:36.323244 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c74085-74e4-46b3-984b-24b0d490f001 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:36.333586 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1402/2832] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:35:36.338400 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acfa1d-97bc-42d2-9d74-5f61cdb8e326 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:36.348116 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acfa1d-97bc-42d2-9d74-5f61cdb8e326 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:36.352741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc958e-2ebd-4699-96d1-b890bb451d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:36.353607 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc958e-2ebd-4699-96d1-b890bb451d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:36.353887 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc958e-2ebd-4699-96d1-b890bb451d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:36.354162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc958e-2ebd-4699-96d1-b890bb451d51 admin admin] PATH_INFO: `/v3/projects/cb330a7af613434090b1faa1a9230a59/users/45d9b287f50e4216b8f1a1451f4123e9/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:36.368993 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc958e-2ebd-4699-96d1-b890bb451d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb330a7af613434090b1faa1a9230a59, user_id=45d9b287f50e4216b8f1a1451f4123e9, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:36.371651 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc958e-2ebd-4699-96d1-b890bb451d51 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:36.419714 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1431/2833] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:35:36 2022] PUT /identity/v3/projects/cb330a7af613434090b1faa1a9230a59/users/45d9b287f50e4216b8f1a1451f4123e9/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:35:36.423586 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c74085-74e4-46b3-984b-24b0d490f001 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:36.434388 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c74085-74e4-46b3-984b-24b0d490f001 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:36.491396 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7d650-528d-41a7-a1ed-1de74598c299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:36.492069 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7d650-528d-41a7-a1ed-1de74598c299 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:36.492352 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7d650-528d-41a7-a1ed-1de74598c299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:36.492627 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7d650-528d-41a7-a1ed-1de74598c299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:36.493403 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7d650-528d-41a7-a1ed-1de74598c299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:36.494955 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7d650-528d-41a7-a1ed-1de74598c299 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:36.504923 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1403/2834] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:35:36.509980 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc958e-2ebd-4699-96d1-b890bb451d51 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:36.520731 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc958e-2ebd-4699-96d1-b890bb451d51 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:36.525652 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cff49e-8f1c-4626-9f9e-a105850ffdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:36.526533 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cff49e-8f1c-4626-9f9e-a105850ffdad admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:36.526814 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cff49e-8f1c-4626-9f9e-a105850ffdad admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:36.527094 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cff49e-8f1c-4626-9f9e-a105850ffdad admin admin] PATH_INFO: `/v3/projects/cb330a7af613434090b1faa1a9230a59/users/45d9b287f50e4216b8f1a1451f4123e9/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:36.529182 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cff49e-8f1c-4626-9f9e-a105850ffdad admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb330a7af613434090b1faa1a9230a59, user_id=45d9b287f50e4216b8f1a1451f4123e9, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:36.531843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cff49e-8f1c-4626-9f9e-a105850ffdad admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:36.542236 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1432/2835] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:35:36 2022] PUT /identity/v3/projects/cb330a7af613434090b1faa1a9230a59/users/45d9b287f50e4216b8f1a1451f4123e9/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:36.602022 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b49800-361e-4a98-b02d-3d3bafa8c007 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:36.602316 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b49800-361e-4a98-b02d-3d3bafa8c007 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:36.602609 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b49800-361e-4a98-b02d-3d3bafa8c007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:36.629729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-18b49800-361e-4a98-b02d-3d3bafa8c007 None None] MFA Rules not processed for user `45d9b287f50e4216b8f1a1451f4123e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:36.630946 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-18b49800-361e-4a98-b02d-3d3bafa8c007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:36.681490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:36.703087 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1404/2836] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:35:36 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:36.714912 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cff49e-8f1c-4626-9f9e-a105850ffdad admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:36.718479 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-33cff49e-8f1c-4626-9f9e-a105850ffdad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:36.722075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:36.774848 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cff49e-8f1c-4626-9f9e-a105850ffdad admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:36.778610 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f460c51-872a-4eca-941e-2ff901e6888b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:36.779300 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f460c51-872a-4eca-941e-2ff901e6888b service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:36.779571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f460c51-872a-4eca-941e-2ff901e6888b service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:36.779850 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f460c51-872a-4eca-941e-2ff901e6888b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:36.791621 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f460c51-872a-4eca-941e-2ff901e6888b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:36.793709 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f460c51-872a-4eca-941e-2ff901e6888b service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:36.804161 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1433/2837] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:35:36 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:35:37.017743 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b49800-361e-4a98-b02d-3d3bafa8c007 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:37.028542 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b49800-361e-4a98-b02d-3d3bafa8c007 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:37.032071 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638667d-f562-4aa1-82be-99ce2b24df73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:37.032708 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638667d-f562-4aa1-82be-99ce2b24df73 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:37.032989 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638667d-f562-4aa1-82be-99ce2b24df73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:37.033264 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638667d-f562-4aa1-82be-99ce2b24df73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:37.091906 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638667d-f562-4aa1-82be-99ce2b24df73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:37.093358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638667d-f562-4aa1-82be-99ce2b24df73 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:37.103026 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1405/2838] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:35:37 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:35:46.837383 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=25 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:47.098493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=35 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:54.236027 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f460c51-872a-4eca-941e-2ff901e6888b service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:54.296449 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f460c51-872a-4eca-941e-2ff901e6888b service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:54.301793 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdfc93-1e0f-44f1-b5ea-94b59b07ed49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:54.302655 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdfc93-1e0f-44f1-b5ea-94b59b07ed49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:54.302937 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdfc93-1e0f-44f1-b5ea-94b59b07ed49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:54.303238 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdfc93-1e0f-44f1-b5ea-94b59b07ed49 admin admin] PATH_INFO: `/v3/users/caec2c1298ab45d3a244112d8775b371` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:54.304426 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfdfc93-1e0f-44f1-b5ea-94b59b07ed49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caec2c1298ab45d3a244112d8775b371)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:54.306320 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfdfc93-1e0f-44f1-b5ea-94b59b07ed49 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:54.484684 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ecfdfc93-1e0f-44f1-b5ea-94b59b07ed49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caec2c1298ab45d3a244112d8775b371'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:54.496185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ecfdfc93-1e0f-44f1-b5ea-94b59b07ed49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caec2c1298ab45d3a244112d8775b371'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:54.567321 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ecfdfc93-1e0f-44f1-b5ea-94b59b07ed49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caec2c1298ab45d3a244112d8775b371'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:54.567944 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ecfdfc93-1e0f-44f1-b5ea-94b59b07ed49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caec2c1298ab45d3a244112d8775b371'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:54.574938 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ecfdfc93-1e0f-44f1-b5ea-94b59b07ed49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caec2c1298ab45d3a244112d8775b371'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:54.581110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1434/2839] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:35:54 2022] DELETE /identity/v3/users/caec2c1298ab45d3a244112d8775b371 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:35:54.585586 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76782a-f646-42c9-8053-4773ac99098a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:54.586100 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76782a-f646-42c9-8053-4773ac99098a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:54.586206 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76782a-f646-42c9-8053-4773ac99098a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:54.603358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ba76782a-f646-42c9-8053-4773ac99098a None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:54.604551 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ba76782a-f646-42c9-8053-4773ac99098a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:54.663085 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1406/2840] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:54.673668 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdfc93-1e0f-44f1-b5ea-94b59b07ed49 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:54.675075 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ecfdfc93-1e0f-44f1-b5ea-94b59b07ed49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:54.730620 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdfc93-1e0f-44f1-b5ea-94b59b07ed49 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:54.734881 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f0312-b16b-48dc-9189-ad0a89d375b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:54.735721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f0312-b16b-48dc-9189-ad0a89d375b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:54.736009 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f0312-b16b-48dc-9189-ad0a89d375b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:54.736285 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f0312-b16b-48dc-9189-ad0a89d375b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:54.745337 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f0312-b16b-48dc-9189-ad0a89d375b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:54.746815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f0312-b16b-48dc-9189-ad0a89d375b2 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:54.756743 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1435/2841] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:35:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:35:55.150727 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76782a-f646-42c9-8053-4773ac99098a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:55.152374 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ba76782a-f646-42c9-8053-4773ac99098a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:55.166049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76782a-f646-42c9-8053-4773ac99098a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:55.173066 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff7e78-a70f-4011-9196-1cd03511c04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:55.173807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff7e78-a70f-4011-9196-1cd03511c04e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:55.174120 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff7e78-a70f-4011-9196-1cd03511c04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:55.174470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff7e78-a70f-4011-9196-1cd03511c04e admin admin] PATH_INFO: `/v3/projects/e482a3fd2ced474d89932ae845d7aa9c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:55.175674 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff7e78-a70f-4011-9196-1cd03511c04e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e482a3fd2ced474d89932ae845d7aa9c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:55.177566 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff7e78-a70f-4011-9196-1cd03511c04e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:55.300246 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7dff7e78-a70f-4011-9196-1cd03511c04e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e482a3fd2ced474d89932ae845d7aa9c', 'request_id': 'req-7dff7e78-a70f-4011-9196-1cd03511c04e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:55.306269 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7dff7e78-a70f-4011-9196-1cd03511c04e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e482a3fd2ced474d89932ae845d7aa9c', 'request_id': 'req-7dff7e78-a70f-4011-9196-1cd03511c04e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:55.373953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7dff7e78-a70f-4011-9196-1cd03511c04e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e482a3fd2ced474d89932ae845d7aa9c', 'request_id': 'req-7dff7e78-a70f-4011-9196-1cd03511c04e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:55.384538 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7dff7e78-a70f-4011-9196-1cd03511c04e admin admin] The token cache is being invalidate because project e482a3fd2ced474d89932ae845d7aa9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:35:55.384826 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7dff7e78-a70f-4011-9196-1cd03511c04e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:55.385444 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7dff7e78-a70f-4011-9196-1cd03511c04e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:55.386982 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1407/2842] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:35:55 2022] DELETE /identity/v3/projects/e482a3fd2ced474d89932ae845d7aa9c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:55.390937 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f0312-b16b-48dc-9189-ad0a89d375b2 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:55.400862 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f0312-b16b-48dc-9189-ad0a89d375b2 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:55.418936 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:55.457471 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1f96a-177b-4903-b2c2-587292f5c3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:55.458354 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1f96a-177b-4903-b2c2-587292f5c3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:55.458643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1f96a-177b-4903-b2c2-587292f5c3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:55.458919 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1f96a-177b-4903-b2c2-587292f5c3fc admin admin] PATH_INFO: `/v3/users/45d9b287f50e4216b8f1a1451f4123e9` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:55.460139 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1f96a-177b-4903-b2c2-587292f5c3fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=45d9b287f50e4216b8f1a1451f4123e9)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:55.462054 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1f96a-177b-4903-b2c2-587292f5c3fc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:55.660886 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2ad1f96a-177b-4903-b2c2-587292f5c3fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45d9b287f50e4216b8f1a1451f4123e9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:55.672819 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2ad1f96a-177b-4903-b2c2-587292f5c3fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45d9b287f50e4216b8f1a1451f4123e9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:55.738497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2ad1f96a-177b-4903-b2c2-587292f5c3fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45d9b287f50e4216b8f1a1451f4123e9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:55.739106 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2ad1f96a-177b-4903-b2c2-587292f5c3fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45d9b287f50e4216b8f1a1451f4123e9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:55.746189 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2ad1f96a-177b-4903-b2c2-587292f5c3fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45d9b287f50e4216b8f1a1451f4123e9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:55.752070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1436/2843] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:35:55 2022] DELETE /identity/v3/users/45d9b287f50e4216b8f1a1451f4123e9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:56.170520 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff7e78-a70f-4011-9196-1cd03511c04e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:56.172245 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7dff7e78-a70f-4011-9196-1cd03511c04e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:56.186310 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff7e78-a70f-4011-9196-1cd03511c04e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:56.245436 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42eb2c-36e8-471c-8e79-1e5874526ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:56.246290 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42eb2c-36e8-471c-8e79-1e5874526ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:56.246571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42eb2c-36e8-471c-8e79-1e5874526ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:56.246844 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42eb2c-36e8-471c-8e79-1e5874526ab6 admin admin] PATH_INFO: `/v3/projects/cb330a7af613434090b1faa1a9230a59` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:56.248057 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f42eb2c-36e8-471c-8e79-1e5874526ab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb330a7af613434090b1faa1a9230a59)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:56.251007 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f42eb2c-36e8-471c-8e79-1e5874526ab6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:56.541590 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:56.558163 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6f42eb2c-36e8-471c-8e79-1e5874526ab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb330a7af613434090b1faa1a9230a59', 'request_id': 'req-6f42eb2c-36e8-471c-8e79-1e5874526ab6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:56.565187 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6f42eb2c-36e8-471c-8e79-1e5874526ab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb330a7af613434090b1faa1a9230a59', 'request_id': 'req-6f42eb2c-36e8-471c-8e79-1e5874526ab6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:56.660502 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6f42eb2c-36e8-471c-8e79-1e5874526ab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb330a7af613434090b1faa1a9230a59', 'request_id': 'req-6f42eb2c-36e8-471c-8e79-1e5874526ab6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:56.672237 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6f42eb2c-36e8-471c-8e79-1e5874526ab6 admin admin] The token cache is being invalidate because project cb330a7af613434090b1faa1a9230a59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:35:56.672807 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6f42eb2c-36e8-471c-8e79-1e5874526ab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:56.673562 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6f42eb2c-36e8-471c-8e79-1e5874526ab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:56.675153 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1408/2844] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:35:56 2022] DELETE /identity/v3/projects/cb330a7af613434090b1faa1a9230a59 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:56.680265 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055c996-1e52-477a-bbc9-851557b57b2d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:56.680796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055c996-1e52-477a-bbc9-851557b57b2d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:56.680796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055c996-1e52-477a-bbc9-851557b57b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:56.699544 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2055c996-1e52-477a-bbc9-851557b57b2d None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:56.700729 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2055c996-1e52-477a-bbc9-851557b57b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:56.759451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1437/2845] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:56.769937 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d9a9e-85b4-429f-a5c9-7364032b2b3c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:56.770227 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d9a9e-85b4-429f-a5c9-7364032b2b3c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:56.770509 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d9a9e-85b4-429f-a5c9-7364032b2b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:56.790772 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-810d9a9e-85b4-429f-a5c9-7364032b2b3c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:56.792049 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-810d9a9e-85b4-429f-a5c9-7364032b2b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:56.800460 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1409/2846] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:56.804718 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055c996-1e52-477a-bbc9-851557b57b2d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:56.814864 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055c996-1e52-477a-bbc9-851557b57b2d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:56.819448 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce74351-f6de-46bc-869a-740e8890176d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:56.820102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce74351-f6de-46bc-869a-740e8890176d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:56.820386 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce74351-f6de-46bc-869a-740e8890176d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:56.820664 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce74351-f6de-46bc-869a-740e8890176d admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:56.821389 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce74351-f6de-46bc-869a-740e8890176d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:35:56.821698 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce74351-f6de-46bc-869a-740e8890176d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:56.823348 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce74351-f6de-46bc-869a-740e8890176d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:56.834689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1438/2847] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:35:56.838860 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d9a9e-85b4-429f-a5c9-7364032b2b3c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:56.848717 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d9a9e-85b4-429f-a5c9-7364032b2b3c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:56.853350 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af87e5-7dc3-402b-bad0-13c5b1ae32ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:56.853995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af87e5-7dc3-402b-bad0-13c5b1ae32ca admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:56.854276 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af87e5-7dc3-402b-bad0-13c5b1ae32ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:56.854550 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af87e5-7dc3-402b-bad0-13c5b1ae32ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:56.855274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af87e5-7dc3-402b-bad0-13c5b1ae32ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:56.857171 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af87e5-7dc3-402b-bad0-13c5b1ae32ca admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:56.936357 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1410/2848] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:35:56 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:35:56.941022 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce74351-f6de-46bc-869a-740e8890176d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:56.951928 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce74351-f6de-46bc-869a-740e8890176d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:56.980681 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:57.018381 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09add5c2-464f-407e-a27e-7e77ccd7808d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:57.019340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09add5c2-464f-407e-a27e-7e77ccd7808d admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:57.019648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09add5c2-464f-407e-a27e-7e77ccd7808d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:57.019924 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09add5c2-464f-407e-a27e-7e77ccd7808d admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:57.020665 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09add5c2-464f-407e-a27e-7e77ccd7808d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:57.022643 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09add5c2-464f-407e-a27e-7e77ccd7808d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:57.134726 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1439/2849] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:35:56 2022] POST /identity/v3/users => generated 294 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:35:57.139035 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af87e5-7dc3-402b-bad0-13c5b1ae32ca admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:57.150570 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af87e5-7dc3-402b-bad0-13c5b1ae32ca admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:57.155134 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd400b-3188-426d-ace7-769fa09bf8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:57.156001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd400b-3188-426d-ace7-769fa09bf8dc admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:57.156330 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd400b-3188-426d-ace7-769fa09bf8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:57.156769 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd400b-3188-426d-ace7-769fa09bf8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:57.157748 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd400b-3188-426d-ace7-769fa09bf8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:57.159644 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd400b-3188-426d-ace7-769fa09bf8dc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:57.170877 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1411/2850] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:35:57.175834 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09add5c2-464f-407e-a27e-7e77ccd7808d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:57.185774 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09add5c2-464f-407e-a27e-7e77ccd7808d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:57.191415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90def353-9819-4726-858e-6e7a58df61b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:57.192106 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90def353-9819-4726-858e-6e7a58df61b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:57.192388 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90def353-9819-4726-858e-6e7a58df61b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:57.192662 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90def353-9819-4726-858e-6e7a58df61b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:57.193375 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90def353-9819-4726-858e-6e7a58df61b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:57.194960 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90def353-9819-4726-858e-6e7a58df61b7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:57.204895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1440/2851] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:35:57.209139 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd400b-3188-426d-ace7-769fa09bf8dc admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:57.219080 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd400b-3188-426d-ace7-769fa09bf8dc admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:57.223760 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7a96b-3cfd-4d24-9bd3-836586ca7a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:57.224649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7a96b-3cfd-4d24-9bd3-836586ca7a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:57.224932 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7a96b-3cfd-4d24-9bd3-836586ca7a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:57.225209 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7a96b-3cfd-4d24-9bd3-836586ca7a87 admin admin] PATH_INFO: `/v3/projects/bb27d46c1e4041939eeaea984316a4ec/users/2461387c48094553a9706e5c8ceca04c/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:57.239926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7a96b-3cfd-4d24-9bd3-836586ca7a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb27d46c1e4041939eeaea984316a4ec, user_id=2461387c48094553a9706e5c8ceca04c, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:57.242563 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7a96b-3cfd-4d24-9bd3-836586ca7a87 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:57.327324 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1412/2852] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:35:57 2022] PUT /identity/v3/projects/bb27d46c1e4041939eeaea984316a4ec/users/2461387c48094553a9706e5c8ceca04c/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:57.339000 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff332a18-4df1-4771-847d-31cffcdd6b6c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:57.339329 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff332a18-4df1-4771-847d-31cffcdd6b6c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:57.339612 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff332a18-4df1-4771-847d-31cffcdd6b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:57.366029 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ff332a18-4df1-4771-847d-31cffcdd6b6c None None] MFA Rules not processed for user `2461387c48094553a9706e5c8ceca04c`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:57.367235 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ff332a18-4df1-4771-847d-31cffcdd6b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:57.429476 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1441/2853] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:35:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:57.433927 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7a96b-3cfd-4d24-9bd3-836586ca7a87 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:57.495863 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7a96b-3cfd-4d24-9bd3-836586ca7a87 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:57.500996 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e7bd2-27a9-4a7a-9c01-a8c93f4381e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:57.501648 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e7bd2-27a9-4a7a-9c01-a8c93f4381e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:57.501934 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e7bd2-27a9-4a7a-9c01-a8c93f4381e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:57.502278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e7bd2-27a9-4a7a-9c01-a8c93f4381e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:57.502961 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e7bd2-27a9-4a7a-9c01-a8c93f4381e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:57.504953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e7bd2-27a9-4a7a-9c01-a8c93f4381e6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:57.626827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1413/2854] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:35:57 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:35:57.630746 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff332a18-4df1-4771-847d-31cffcdd6b6c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:57.641934 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff332a18-4df1-4771-847d-31cffcdd6b6c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:57.696734 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d584f3-a990-4bee-b807-cf409552f7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:57.697420 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d584f3-a990-4bee-b807-cf409552f7dc admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:57.697670 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d584f3-a990-4bee-b807-cf409552f7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:57.697948 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d584f3-a990-4bee-b807-cf409552f7dc admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:57.698629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d584f3-a990-4bee-b807-cf409552f7dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:57.700578 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d584f3-a990-4bee-b807-cf409552f7dc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:57.779415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1442/2855] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:35:57 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:35:57.783662 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e7bd2-27a9-4a7a-9c01-a8c93f4381e6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:57.793780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e7bd2-27a9-4a7a-9c01-a8c93f4381e6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:57.798854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4974079-4a45-42c4-85a8-805833912314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:57.799592 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4974079-4a45-42c4-85a8-805833912314 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:57.799881 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4974079-4a45-42c4-85a8-805833912314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:57.800164 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4974079-4a45-42c4-85a8-805833912314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:57.800897 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4974079-4a45-42c4-85a8-805833912314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:57.802555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4974079-4a45-42c4-85a8-805833912314 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:57.813611 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1414/2856] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:35:57.817870 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d584f3-a990-4bee-b807-cf409552f7dc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:57.827840 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d584f3-a990-4bee-b807-cf409552f7dc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:57.833400 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96d169-6bf4-41b8-ba38-e51f14d0af3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:57.834276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96d169-6bf4-41b8-ba38-e51f14d0af3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:57.834558 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96d169-6bf4-41b8-ba38-e51f14d0af3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:57.834836 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96d169-6bf4-41b8-ba38-e51f14d0af3b admin admin] PATH_INFO: `/v3/projects/4bc9e7ca6fa7416f8b7dcd234d99e690/users/80773f1f0521415fa4c7f1f682640686/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:57.849045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96d169-6bf4-41b8-ba38-e51f14d0af3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc9e7ca6fa7416f8b7dcd234d99e690, user_id=80773f1f0521415fa4c7f1f682640686, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:57.851725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96d169-6bf4-41b8-ba38-e51f14d0af3b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:57.915608 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1443/2857] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:35:57 2022] PUT /identity/v3/projects/4bc9e7ca6fa7416f8b7dcd234d99e690/users/80773f1f0521415fa4c7f1f682640686/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:57.919930 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4974079-4a45-42c4-85a8-805833912314 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:57.930478 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4974079-4a45-42c4-85a8-805833912314 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:57.950741 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:57.986361 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300593cd-cf60-4bfb-bd6f-c14d91d87bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:57.987020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300593cd-cf60-4bfb-bd6f-c14d91d87bff admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:57.987348 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300593cd-cf60-4bfb-bd6f-c14d91d87bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:57.987694 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300593cd-cf60-4bfb-bd6f-c14d91d87bff admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:57.988432 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300593cd-cf60-4bfb-bd6f-c14d91d87bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:57.990081 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300593cd-cf60-4bfb-bd6f-c14d91d87bff admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:57.999585 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1415/2858] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:35:58.004683 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96d169-6bf4-41b8-ba38-e51f14d0af3b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:58.010046 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:58.015306 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96d169-6bf4-41b8-ba38-e51f14d0af3b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:58.020476 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a35d1c-4707-4502-a887-447751b0737c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:58.021351 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a35d1c-4707-4502-a887-447751b0737c admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:58.021631 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a35d1c-4707-4502-a887-447751b0737c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:58.022024 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a35d1c-4707-4502-a887-447751b0737c admin admin] PATH_INFO: `/v3/projects/4bc9e7ca6fa7416f8b7dcd234d99e690/users/80773f1f0521415fa4c7f1f682640686/roles/6f8dcd4706ea44f1bcdcfd97ac90878d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:58.023997 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a35d1c-4707-4502-a887-447751b0737c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc9e7ca6fa7416f8b7dcd234d99e690, user_id=80773f1f0521415fa4c7f1f682640686, role_id=6f8dcd4706ea44f1bcdcfd97ac90878d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:58.026602 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a35d1c-4707-4502-a887-447751b0737c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:58.036946 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1444/2859] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:35:58 2022] PUT /identity/v3/projects/4bc9e7ca6fa7416f8b7dcd234d99e690/users/80773f1f0521415fa4c7f1f682640686/roles/6f8dcd4706ea44f1bcdcfd97ac90878d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:58.047820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16ae7a-2cf5-4d2d-90f9-1c153aef9809 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:58.048225 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16ae7a-2cf5-4d2d-90f9-1c153aef9809 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:58.048521 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16ae7a-2cf5-4d2d-90f9-1c153aef9809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:58.074767 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3f16ae7a-2cf5-4d2d-90f9-1c153aef9809 None None] MFA Rules not processed for user `80773f1f0521415fa4c7f1f682640686`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:58.076045 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3f16ae7a-2cf5-4d2d-90f9-1c153aef9809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:58.144729 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1416/2860] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:35:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:58.156994 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a35d1c-4707-4502-a887-447751b0737c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:58.158712 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a6a35d1c-4707-4502-a887-447751b0737c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:58.216172 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a35d1c-4707-4502-a887-447751b0737c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:58.220289 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a3e4a-ca7a-40e2-a080-1c1870cba229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:58.220922 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a3e4a-ca7a-40e2-a080-1c1870cba229 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:58.221202 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a3e4a-ca7a-40e2-a080-1c1870cba229 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:58.221483 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a3e4a-ca7a-40e2-a080-1c1870cba229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:58.230352 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a3e4a-ca7a-40e2-a080-1c1870cba229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:58.231817 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a3e4a-ca7a-40e2-a080-1c1870cba229 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:58.241061 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1445/2861] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:35:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:35:58.270331 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16ae7a-2cf5-4d2d-90f9-1c153aef9809 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:58.333155 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16ae7a-2cf5-4d2d-90f9-1c153aef9809 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:58.337682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cd005-5511-4b0a-9a1c-2881168149d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:58.338572 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cd005-5511-4b0a-9a1c-2881168149d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:58.338857 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cd005-5511-4b0a-9a1c-2881168149d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:58.339136 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cd005-5511-4b0a-9a1c-2881168149d9 admin admin] PATH_INFO: `/v3/users/2461387c48094553a9706e5c8ceca04c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:58.340366 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5cd005-5511-4b0a-9a1c-2881168149d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2461387c48094553a9706e5c8ceca04c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:58.342273 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5cd005-5511-4b0a-9a1c-2881168149d9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:58.536091 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ff5cd005-5511-4b0a-9a1c-2881168149d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2461387c48094553a9706e5c8ceca04c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:58.537157 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:58.548618 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ff5cd005-5511-4b0a-9a1c-2881168149d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2461387c48094553a9706e5c8ceca04c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:58.549334 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ff5cd005-5511-4b0a-9a1c-2881168149d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2461387c48094553a9706e5c8ceca04c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:58.555960 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ff5cd005-5511-4b0a-9a1c-2881168149d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2461387c48094553a9706e5c8ceca04c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:58.560144 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ff5cd005-5511-4b0a-9a1c-2881168149d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2461387c48094553a9706e5c8ceca04c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:58.648253 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1417/2862] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:35:58 2022] DELETE /identity/v3/users/2461387c48094553a9706e5c8ceca04c => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:58.652679 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105dd7f-faa8-4eba-88ef-d9ee688d3e90 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:58.652972 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105dd7f-faa8-4eba-88ef-d9ee688d3e90 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:58.653248 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105dd7f-faa8-4eba-88ef-d9ee688d3e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:58.670843 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1105dd7f-faa8-4eba-88ef-d9ee688d3e90 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:58.672062 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1105dd7f-faa8-4eba-88ef-d9ee688d3e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:58.728384 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1446/2863] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:58.737823 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cd005-5511-4b0a-9a1c-2881168149d9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:58.739216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ff5cd005-5511-4b0a-9a1c-2881168149d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:58.794975 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cd005-5511-4b0a-9a1c-2881168149d9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:58.798675 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d2eef-a97b-4a3f-b1a7-68dc8e47d91e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:58.799351 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d2eef-a97b-4a3f-b1a7-68dc8e47d91e service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:58.799635 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d2eef-a97b-4a3f-b1a7-68dc8e47d91e service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:58.799910 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d2eef-a97b-4a3f-b1a7-68dc8e47d91e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:58.808678 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d2eef-a97b-4a3f-b1a7-68dc8e47d91e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:58.810094 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d2eef-a97b-4a3f-b1a7-68dc8e47d91e service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:58.819463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1418/2864] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:35:59.232223 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105dd7f-faa8-4eba-88ef-d9ee688d3e90 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:59.233747 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1105dd7f-faa8-4eba-88ef-d9ee688d3e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:59.246497 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105dd7f-faa8-4eba-88ef-d9ee688d3e90 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:59.252276 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:59.253341 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:59.253644 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:59.253906 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3 admin admin] PATH_INFO: `/v3/projects/bb27d46c1e4041939eeaea984316a4ec` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:59.255185 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb27d46c1e4041939eeaea984316a4ec)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:59.257203 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:59.350053 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb27d46c1e4041939eeaea984316a4ec', 'request_id': 'req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:59.356081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb27d46c1e4041939eeaea984316a4ec', 'request_id': 'req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:59.402466 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb27d46c1e4041939eeaea984316a4ec', 'request_id': 'req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:59.413259 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3 admin admin] The token cache is being invalidate because project bb27d46c1e4041939eeaea984316a4ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:35:59.413551 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:59.414181 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:59.415847 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1447/2865] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:35:59 2022] DELETE /identity/v3/projects/bb27d46c1e4041939eeaea984316a4ec => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:59.419882 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d2eef-a97b-4a3f-b1a7-68dc8e47d91e service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:59.430132 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d2eef-a97b-4a3f-b1a7-68dc8e47d91e service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:59.485571 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2ff2b-63d2-44c5-b9fc-11067d3d501a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:59.486443 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2ff2b-63d2-44c5-b9fc-11067d3d501a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:59.486733 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2ff2b-63d2-44c5-b9fc-11067d3d501a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:59.487008 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2ff2b-63d2-44c5-b9fc-11067d3d501a admin admin] PATH_INFO: `/v3/users/80773f1f0521415fa4c7f1f682640686` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:59.488207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2ff2b-63d2-44c5-b9fc-11067d3d501a admin admin] RBAC: Authorizing `identity:delete_user(user_id=80773f1f0521415fa4c7f1f682640686)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:59.490121 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2ff2b-63d2-44c5-b9fc-11067d3d501a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:59.665899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-27c2ff2b-63d2-44c5-b9fc-11067d3d501a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80773f1f0521415fa4c7f1f682640686'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:59.678516 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-27c2ff2b-63d2-44c5-b9fc-11067d3d501a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80773f1f0521415fa4c7f1f682640686'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:59.679205 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-27c2ff2b-63d2-44c5-b9fc-11067d3d501a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80773f1f0521415fa4c7f1f682640686'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:59.683580 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:59.686795 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-27c2ff2b-63d2-44c5-b9fc-11067d3d501a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80773f1f0521415fa4c7f1f682640686'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:59.688186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-27c2ff2b-63d2-44c5-b9fc-11067d3d501a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80773f1f0521415fa4c7f1f682640686'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:59.773020 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1419/2866] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:35:59 2022] DELETE /identity/v3/users/80773f1f0521415fa4c7f1f682640686 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:36:00.169940 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:00.171444 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:00.185329 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c93aaa-c95d-4c1d-a562-8ee03fdb7ba3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:00.198154 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:00.242948 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e519a-7424-49bb-a121-ed27ae90e995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:00.243854 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e519a-7424-49bb-a121-ed27ae90e995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:00.244141 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e519a-7424-49bb-a121-ed27ae90e995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:00.244421 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e519a-7424-49bb-a121-ed27ae90e995 admin admin] PATH_INFO: `/v3/projects/4bc9e7ca6fa7416f8b7dcd234d99e690` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:00.245619 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e519a-7424-49bb-a121-ed27ae90e995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bc9e7ca6fa7416f8b7dcd234d99e690)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:00.247575 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e519a-7424-49bb-a121-ed27ae90e995 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:00.384155 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ef3e519a-7424-49bb-a121-ed27ae90e995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc9e7ca6fa7416f8b7dcd234d99e690', 'request_id': 'req-ef3e519a-7424-49bb-a121-ed27ae90e995'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:00.390856 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ef3e519a-7424-49bb-a121-ed27ae90e995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc9e7ca6fa7416f8b7dcd234d99e690', 'request_id': 'req-ef3e519a-7424-49bb-a121-ed27ae90e995'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:00.453440 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ef3e519a-7424-49bb-a121-ed27ae90e995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc9e7ca6fa7416f8b7dcd234d99e690', 'request_id': 'req-ef3e519a-7424-49bb-a121-ed27ae90e995'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:00.463908 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ef3e519a-7424-49bb-a121-ed27ae90e995 admin admin] The token cache is being invalidate because project 4bc9e7ca6fa7416f8b7dcd234d99e690 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:36:00.464194 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ef3e519a-7424-49bb-a121-ed27ae90e995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:00.464815 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ef3e519a-7424-49bb-a121-ed27ae90e995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:00.466345 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1448/2867] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:36:00 2022] DELETE /identity/v3/projects/4bc9e7ca6fa7416f8b7dcd234d99e690 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:36:00.472764 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffab2ab-0eff-4187-bc2f-a6a5ec58e07c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:00.473182 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffab2ab-0eff-4187-bc2f-a6a5ec58e07c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:00.473358 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffab2ab-0eff-4187-bc2f-a6a5ec58e07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:00.491544 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3ffab2ab-0eff-4187-bc2f-a6a5ec58e07c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:00.492488 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3ffab2ab-0eff-4187-bc2f-a6a5ec58e07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:00.553236 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1420/2868] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:00.561208 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c5566-237b-41c0-8163-ef9bdf2de05a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:00.561499 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c5566-237b-41c0-8163-ef9bdf2de05a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:00.561779 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c5566-237b-41c0-8163-ef9bdf2de05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:00.704057 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5d3c5566-237b-41c0-8163-ef9bdf2de05a None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:00.705180 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5d3c5566-237b-41c0-8163-ef9bdf2de05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:00.713164 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1449/2869] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:00.717493 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffab2ab-0eff-4187-bc2f-a6a5ec58e07c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:00.727224 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffab2ab-0eff-4187-bc2f-a6a5ec58e07c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:00.731929 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afd6c5-02cc-46fb-8c69-e1f8e8cdde71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:00.732588 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afd6c5-02cc-46fb-8c69-e1f8e8cdde71 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:00.732870 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afd6c5-02cc-46fb-8c69-e1f8e8cdde71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:00.733145 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afd6c5-02cc-46fb-8c69-e1f8e8cdde71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:00.733868 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afd6c5-02cc-46fb-8c69-e1f8e8cdde71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:36:00.734199 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afd6c5-02cc-46fb-8c69-e1f8e8cdde71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:00.735818 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afd6c5-02cc-46fb-8c69-e1f8e8cdde71 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:00.747218 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1421/2870] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:36:00.751989 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c5566-237b-41c0-8163-ef9bdf2de05a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:00.760896 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c5566-237b-41c0-8163-ef9bdf2de05a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:00.766080 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af3bbb-c3dd-4401-b84e-6aca43c209da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:00.766650 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af3bbb-c3dd-4401-b84e-6aca43c209da admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:00.766931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af3bbb-c3dd-4401-b84e-6aca43c209da admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:00.767229 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af3bbb-c3dd-4401-b84e-6aca43c209da admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:00.767915 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af3bbb-c3dd-4401-b84e-6aca43c209da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:00.770045 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af3bbb-c3dd-4401-b84e-6aca43c209da admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:00.849503 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1450/2871] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:36:00 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:36:00.853745 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afd6c5-02cc-46fb-8c69-e1f8e8cdde71 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:00.863693 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afd6c5-02cc-46fb-8c69-e1f8e8cdde71 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:00.918995 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bc9f0-7227-4c97-b4ed-8b13eb68eec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:00.919682 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bc9f0-7227-4c97-b4ed-8b13eb68eec6 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:00.919963 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bc9f0-7227-4c97-b4ed-8b13eb68eec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:00.920233 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bc9f0-7227-4c97-b4ed-8b13eb68eec6 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:00.920899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8bc9f0-7227-4c97-b4ed-8b13eb68eec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:00.923442 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8bc9f0-7227-4c97-b4ed-8b13eb68eec6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:01.014013 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1422/2872] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:36:00 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:36:01.018260 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af3bbb-c3dd-4401-b84e-6aca43c209da admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:01.028816 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af3bbb-c3dd-4401-b84e-6aca43c209da admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:01.033523 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586272cc-2ae9-4028-992f-cd824bc31e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:01.034173 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586272cc-2ae9-4028-992f-cd824bc31e70 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:01.034451 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586272cc-2ae9-4028-992f-cd824bc31e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:01.034721 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586272cc-2ae9-4028-992f-cd824bc31e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:01.035508 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586272cc-2ae9-4028-992f-cd824bc31e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:01.037070 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586272cc-2ae9-4028-992f-cd824bc31e70 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:01.047209 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1451/2873] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:36:01.051399 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bc9f0-7227-4c97-b4ed-8b13eb68eec6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:01.061247 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bc9f0-7227-4c97-b4ed-8b13eb68eec6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:01.066834 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f617b90-5171-4aaf-943d-3b78d2b531c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:01.067531 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f617b90-5171-4aaf-943d-3b78d2b531c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:01.067820 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f617b90-5171-4aaf-943d-3b78d2b531c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:01.068099 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f617b90-5171-4aaf-943d-3b78d2b531c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:01.068816 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f617b90-5171-4aaf-943d-3b78d2b531c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:01.070393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f617b90-5171-4aaf-943d-3b78d2b531c6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:01.080370 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1423/2874] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:36:01.084304 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586272cc-2ae9-4028-992f-cd824bc31e70 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:01.095081 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586272cc-2ae9-4028-992f-cd824bc31e70 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:01.099813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06352abf-95e8-40f6-ab5f-f63c84bc59ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:01.100696 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06352abf-95e8-40f6-ab5f-f63c84bc59ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:01.100982 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06352abf-95e8-40f6-ab5f-f63c84bc59ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:01.101283 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06352abf-95e8-40f6-ab5f-f63c84bc59ce admin admin] PATH_INFO: `/v3/projects/670bc3376a084590967ec1ff37e20285/users/30bf169113e14550b9f2f95375d9c02e/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:01.116554 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06352abf-95e8-40f6-ab5f-f63c84bc59ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=670bc3376a084590967ec1ff37e20285, user_id=30bf169113e14550b9f2f95375d9c02e, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:01.119216 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06352abf-95e8-40f6-ab5f-f63c84bc59ce admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:01.220201 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1452/2875] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:36:01 2022] PUT /identity/v3/projects/670bc3376a084590967ec1ff37e20285/users/30bf169113e14550b9f2f95375d9c02e/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:36:01.231833 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119c971-293d-4e81-9992-b02948fdb9ae None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:01.232125 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119c971-293d-4e81-9992-b02948fdb9ae None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:01.232412 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119c971-293d-4e81-9992-b02948fdb9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:01.259664 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c119c971-293d-4e81-9992-b02948fdb9ae None None] MFA Rules not processed for user `30bf169113e14550b9f2f95375d9c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:01.260874 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c119c971-293d-4e81-9992-b02948fdb9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:01.303583 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:01.326088 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1424/2876] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:36:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:01.338076 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06352abf-95e8-40f6-ab5f-f63c84bc59ce admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:01.339796 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-06352abf-95e8-40f6-ab5f-f63c84bc59ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:01.386110 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:01.418526 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06352abf-95e8-40f6-ab5f-f63c84bc59ce admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:01.426322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a66955-44d9-4bcf-9c3b-773fdf34339a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1971c464293f426f97d6cb199e48193b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:01.426322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a66955-44d9-4bcf-9c3b-773fdf34339a service glance] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:01.426322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a66955-44d9-4bcf-9c3b-773fdf34339a service glance] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:01.426322 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a66955-44d9-4bcf-9c3b-773fdf34339a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:01.433354 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a66955-44d9-4bcf-9c3b-773fdf34339a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:01.434813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a66955-44d9-4bcf-9c3b-773fdf34339a service glance] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:01.443931 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1453/2877] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:36:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:36:11.362445 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=8 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:11.471127 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=16 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:29.453530 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119c971-293d-4e81-9992-b02948fdb9ae None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:29.514649 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119c971-293d-4e81-9992-b02948fdb9ae None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:29.519446 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c9b32-1d2c-4761-974b-2e0564682bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:29.520314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c9b32-1d2c-4761-974b-2e0564682bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:29.520599 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c9b32-1d2c-4761-974b-2e0564682bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:29.520878 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c9b32-1d2c-4761-974b-2e0564682bc4 admin admin] PATH_INFO: `/v3/users/30bf169113e14550b9f2f95375d9c02e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:29.522116 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c9b32-1d2c-4761-974b-2e0564682bc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30bf169113e14550b9f2f95375d9c02e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:29.524059 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c9b32-1d2c-4761-974b-2e0564682bc4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:29.711897 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-af8c9b32-1d2c-4761-974b-2e0564682bc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30bf169113e14550b9f2f95375d9c02e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:29.723587 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-af8c9b32-1d2c-4761-974b-2e0564682bc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30bf169113e14550b9f2f95375d9c02e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:29.727971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-af8c9b32-1d2c-4761-974b-2e0564682bc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30bf169113e14550b9f2f95375d9c02e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:29.732178 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-af8c9b32-1d2c-4761-974b-2e0564682bc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30bf169113e14550b9f2f95375d9c02e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:29.732827 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-af8c9b32-1d2c-4761-974b-2e0564682bc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30bf169113e14550b9f2f95375d9c02e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:29.795650 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1425/2878] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:36:29 2022] DELETE /identity/v3/users/30bf169113e14550b9f2f95375d9c02e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:36:29.800159 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b506f0-9693-4ae4-ad3c-7549856f0d9c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:29.800702 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b506f0-9693-4ae4-ad3c-7549856f0d9c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:29.800892 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b506f0-9693-4ae4-ad3c-7549856f0d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:29.817628 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-25b506f0-9693-4ae4-ad3c-7549856f0d9c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:29.821468 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-25b506f0-9693-4ae4-ad3c-7549856f0d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:29.876799 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1454/2879] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:29.887119 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c9b32-1d2c-4761-974b-2e0564682bc4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:29.888564 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-af8c9b32-1d2c-4761-974b-2e0564682bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:29.946854 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c9b32-1d2c-4761-974b-2e0564682bc4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:29.950499 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fde16-f4ae-4766-b206-429c55a1226a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:29.951147 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fde16-f4ae-4766-b206-429c55a1226a service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:29.951470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fde16-f4ae-4766-b206-429c55a1226a service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:29.951751 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fde16-f4ae-4766-b206-429c55a1226a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:29.961485 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fde16-f4ae-4766-b206-429c55a1226a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:29.962971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fde16-f4ae-4766-b206-429c55a1226a service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:29.972368 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1426/2880] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:36:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:36:30.369630 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b506f0-9693-4ae4-ad3c-7549856f0d9c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:30.373176 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-25b506f0-9693-4ae4-ad3c-7549856f0d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:30.386124 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b506f0-9693-4ae4-ad3c-7549856f0d9c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:30.393302 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e48022-e286-494d-aae1-89334d9456c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:30.394458 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e48022-e286-494d-aae1-89334d9456c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:30.394741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e48022-e286-494d-aae1-89334d9456c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:30.395571 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e48022-e286-494d-aae1-89334d9456c8 admin admin] PATH_INFO: `/v3/projects/670bc3376a084590967ec1ff37e20285` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:30.396534 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e48022-e286-494d-aae1-89334d9456c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=670bc3376a084590967ec1ff37e20285)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:30.398487 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e48022-e286-494d-aae1-89334d9456c8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:30.498102 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b2e48022-e286-494d-aae1-89334d9456c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670bc3376a084590967ec1ff37e20285', 'request_id': 'req-b2e48022-e286-494d-aae1-89334d9456c8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:30.504735 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b2e48022-e286-494d-aae1-89334d9456c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670bc3376a084590967ec1ff37e20285', 'request_id': 'req-b2e48022-e286-494d-aae1-89334d9456c8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:30.567541 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b2e48022-e286-494d-aae1-89334d9456c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670bc3376a084590967ec1ff37e20285', 'request_id': 'req-b2e48022-e286-494d-aae1-89334d9456c8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:30.579200 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b2e48022-e286-494d-aae1-89334d9456c8 admin admin] The token cache is being invalidate because project 670bc3376a084590967ec1ff37e20285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:36:30.579492 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b2e48022-e286-494d-aae1-89334d9456c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:30.580118 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b2e48022-e286-494d-aae1-89334d9456c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:30.581646 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1455/2881] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:36:30 2022] DELETE /identity/v3/projects/670bc3376a084590967ec1ff37e20285 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:36:30.587031 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc3b4e-2c84-4659-a7fc-c07a6dd7d8d3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:30.587577 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc3b4e-2c84-4659-a7fc-c07a6dd7d8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:30.587691 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc3b4e-2c84-4659-a7fc-c07a6dd7d8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:30.605463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b6cc3b4e-2c84-4659-a7fc-c07a6dd7d8d3 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:30.606716 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b6cc3b4e-2c84-4659-a7fc-c07a6dd7d8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:30.624936 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:30.668500 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1427/2882] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:36:30.676196 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af430344-2ef1-4bca-9fc0-59ffe3409005 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:30.676485 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af430344-2ef1-4bca-9fc0-59ffe3409005 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:30.676764 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af430344-2ef1-4bca-9fc0-59ffe3409005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:30.695648 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-af430344-2ef1-4bca-9fc0-59ffe3409005 None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:30.696866 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-af430344-2ef1-4bca-9fc0-59ffe3409005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:30.704593 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1456/2883] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:30.708677 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc3b4e-2c84-4659-a7fc-c07a6dd7d8d3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:30.718490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc3b4e-2c84-4659-a7fc-c07a6dd7d8d3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:30.723058 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d868321-90f9-4677-881b-96e4b53dd482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:30.723746 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d868321-90f9-4677-881b-96e4b53dd482 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:30.724036 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d868321-90f9-4677-881b-96e4b53dd482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:30.724843 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d868321-90f9-4677-881b-96e4b53dd482 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:30.725847 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d868321-90f9-4677-881b-96e4b53dd482 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:36:30.726166 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d868321-90f9-4677-881b-96e4b53dd482 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:30.727794 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d868321-90f9-4677-881b-96e4b53dd482 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:30.741744 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1428/2884] 10.222.0.67 () {58 vars in 1152 bytes} [Fri Jun 24 15:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:36:30.745860 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af430344-2ef1-4bca-9fc0-59ffe3409005 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:30.754846 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af430344-2ef1-4bca-9fc0-59ffe3409005 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:30.759587 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af6e92-938d-4028-9a54-e8a3741cdff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:30.760225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af6e92-938d-4028-9a54-e8a3741cdff2 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:30.760505 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af6e92-938d-4028-9a54-e8a3741cdff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:30.760782 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af6e92-938d-4028-9a54-e8a3741cdff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:30.761456 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af6e92-938d-4028-9a54-e8a3741cdff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:30.763340 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af6e92-938d-4028-9a54-e8a3741cdff2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:30.847568 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1457/2885] 10.222.0.67 () {60 vars in 1152 bytes} [Fri Jun 24 15:36:30 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:36:30.852207 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d868321-90f9-4677-881b-96e4b53dd482 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:30.862211 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d868321-90f9-4677-881b-96e4b53dd482 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:30.919032 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6470e6-a253-4633-ad1a-f8614f4bb143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:30.919735 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6470e6-a253-4633-ad1a-f8614f4bb143 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:30.920011 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6470e6-a253-4633-ad1a-f8614f4bb143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:30.920289 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6470e6-a253-4633-ad1a-f8614f4bb143 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:30.920988 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6470e6-a253-4633-ad1a-f8614f4bb143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:30.922904 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6470e6-a253-4633-ad1a-f8614f4bb143 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:31.002278 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1429/2886] 10.222.0.67 () {60 vars in 1143 bytes} [Fri Jun 24 15:36:30 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:36:31.007562 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af6e92-938d-4028-9a54-e8a3741cdff2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:31.017769 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af6e92-938d-4028-9a54-e8a3741cdff2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:31.023482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e0fce-91f0-4678-8f3c-38ac130ea8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:31.024162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e0fce-91f0-4678-8f3c-38ac130ea8cf admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:31.024447 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e0fce-91f0-4678-8f3c-38ac130ea8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:31.024725 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e0fce-91f0-4678-8f3c-38ac130ea8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:31.025444 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e0fce-91f0-4678-8f3c-38ac130ea8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:31.027028 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e0fce-91f0-4678-8f3c-38ac130ea8cf admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:31.037599 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1458/2887] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:36:31.041456 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6470e6-a253-4633-ad1a-f8614f4bb143 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:31.051555 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6470e6-a253-4633-ad1a-f8614f4bb143 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:31.056216 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d672e88-5aaf-4c20-980e-adb218e45640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:31.056869 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d672e88-5aaf-4c20-980e-adb218e45640 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:31.057150 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d672e88-5aaf-4c20-980e-adb218e45640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:31.057426 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d672e88-5aaf-4c20-980e-adb218e45640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:31.058137 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d672e88-5aaf-4c20-980e-adb218e45640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:31.059781 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d672e88-5aaf-4c20-980e-adb218e45640 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:31.069964 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1430/2888] 10.222.0.67 () {58 vars in 1121 bytes} [Fri Jun 24 15:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:36:31.073996 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e0fce-91f0-4678-8f3c-38ac130ea8cf admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:31.086951 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e0fce-91f0-4678-8f3c-38ac130ea8cf admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:31.101876 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96df4c-9d52-40d4-81c0-941d99bb4f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:31.102771 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96df4c-9d52-40d4-81c0-941d99bb4f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:31.103058 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96df4c-9d52-40d4-81c0-941d99bb4f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:31.103366 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96df4c-9d52-40d4-81c0-941d99bb4f15 admin admin] PATH_INFO: `/v3/projects/2c75120127524a4aa9c97a3032c1dd2c/users/212357bccadb4604a7d2855c9be77f75/roles/b6dd71d4043d43e9823ec25176863347` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:31.119591 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96df4c-9d52-40d4-81c0-941d99bb4f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c75120127524a4aa9c97a3032c1dd2c, user_id=212357bccadb4604a7d2855c9be77f75, role_id=b6dd71d4043d43e9823ec25176863347)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:31.122290 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96df4c-9d52-40d4-81c0-941d99bb4f15 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:31.202806 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1459/2889] 10.222.0.67 () {60 vars in 1482 bytes} [Fri Jun 24 15:36:31 2022] PUT /identity/v3/projects/2c75120127524a4aa9c97a3032c1dd2c/users/212357bccadb4604a7d2855c9be77f75/roles/b6dd71d4043d43e9823ec25176863347 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:36:31.213463 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd06b4-0200-4fa1-8c44-60052dcae43b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:31.213756 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd06b4-0200-4fa1-8c44-60052dcae43b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:31.214039 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd06b4-0200-4fa1-8c44-60052dcae43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:31.240725 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f8fd06b4-0200-4fa1-8c44-60052dcae43b None None] MFA Rules not processed for user `212357bccadb4604a7d2855c9be77f75`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:31.241918 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f8fd06b4-0200-4fa1-8c44-60052dcae43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:31.305700 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1431/2890] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:36:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:36:31.316336 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96df4c-9d52-40d4-81c0-941d99bb4f15 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:31.318047 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8e96df4c-9d52-40d4-81c0-941d99bb4f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:31.340257 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:31.375869 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96df4c-9d52-40d4-81c0-941d99bb4f15 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:31.379522 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8744762-f40c-48aa-8f67-f1edd00959b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efb120c084a48858d32f97843d76834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:31.380162 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8744762-f40c-48aa-8f67-f1edd00959b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:31.380462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8744762-f40c-48aa-8f67-f1edd00959b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:31.380741 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8744762-f40c-48aa-8f67-f1edd00959b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:31.388929 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8744762-f40c-48aa-8f67-f1edd00959b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:31.390424 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8744762-f40c-48aa-8f67-f1edd00959b0 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:31.407823 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1460/2891] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:36:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:36:33.106461 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b83df3-4f17-4084-8fac-9e713e9c8758 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:33.106965 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b83df3-4f17-4084-8fac-9e713e9c8758 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:33.107086 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b83df3-4f17-4084-8fac-9e713e9c8758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:33.125379 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c6b83df3-4f17-4084-8fac-9e713e9c8758 None None] MFA Rules not processed for user `212357bccadb4604a7d2855c9be77f75`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:33.126583 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c6b83df3-4f17-4084-8fac-9e713e9c8758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:33.133953 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1432/2892] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:36:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:33.655549 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8744762-f40c-48aa-8f67-f1edd00959b0 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:33.657006 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e8744762-f40c-48aa-8f67-f1edd00959b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:33.731720 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8744762-f40c-48aa-8f67-f1edd00959b0 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:33.736895 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93ec27-30d6-49a9-93eb-76c057587b20 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:33.737535 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93ec27-30d6-49a9-93eb-76c057587b20 service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:33.737820 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93ec27-30d6-49a9-93eb-76c057587b20 service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:33.738097 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93ec27-30d6-49a9-93eb-76c057587b20 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:33.739973 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ff93ec27-30d6-49a9-93eb-76c057587b20 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:33.752421 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93ec27-30d6-49a9-93eb-76c057587b20 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:33.753999 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93ec27-30d6-49a9-93eb-76c057587b20 service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:33.763959 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1461/2893] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:36:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:36:37.659964 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b83df3-4f17-4084-8fac-9e713e9c8758 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:37.659964 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c6b83df3-4f17-4084-8fac-9e713e9c8758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:37.675438 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:37.746106 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b83df3-4f17-4084-8fac-9e713e9c8758 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:37.752714 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0f6e9-4ad9-4f40-a44f-c66a83152e84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:37.753393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f6e9-4ad9-4f40-a44f-c66a83152e84 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:37.753681 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f6e9-4ad9-4f40-a44f-c66a83152e84 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:37.753971 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f6e9-4ad9-4f40-a44f-c66a83152e84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:37.755821 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-22a0f6e9-4ad9-4f40-a44f-c66a83152e84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:37.772079 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0f6e9-4ad9-4f40-a44f-c66a83152e84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:37.774629 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0f6e9-4ad9-4f40-a44f-c66a83152e84 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:37.791911 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1433/2894] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:36:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:36:43.759742 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=49 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:44.059309 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93ec27-30d6-49a9-93eb-76c057587b20 service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:44.072066 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93ec27-30d6-49a9-93eb-76c057587b20 service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:44.077466 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15174dbf-e300-451f-a82c-6af0d3b101b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51de59caa3144a1cbf12b20b15e2134f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:44.078109 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15174dbf-e300-451f-a82c-6af0d3b101b8 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:44.078392 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15174dbf-e300-451f-a82c-6af0d3b101b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:44.078670 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15174dbf-e300-451f-a82c-6af0d3b101b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:44.080431 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-15174dbf-e300-451f-a82c-6af0d3b101b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:44.092945 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15174dbf-e300-451f-a82c-6af0d3b101b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:44.094579 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15174dbf-e300-451f-a82c-6af0d3b101b8 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:44.105813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1462/2895] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:36:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:36:47.782272 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=29 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:50.786297 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e21eb-6e84-47a3-9386-8a1e4487706e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:50.786297 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e21eb-6e84-47a3-9386-8a1e4487706e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:50.786297 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e21eb-6e84-47a3-9386-8a1e4487706e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:50.843719 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-036e21eb-6e84-47a3-9386-8a1e4487706e None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:50.844928 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-036e21eb-6e84-47a3-9386-8a1e4487706e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:50.909251 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1434/2896] 10.222.0.67 () {58 vars in 1009 bytes} [Fri Jun 24 15:36:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:51.058152 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15174dbf-e300-451f-a82c-6af0d3b101b8 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:51.059587 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-15174dbf-e300-451f-a82c-6af0d3b101b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:51.152249 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15174dbf-e300-451f-a82c-6af0d3b101b8 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:51.157294 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926bb9f-f1a0-4c32-b227-33aee2f66c06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:51.157968 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926bb9f-f1a0-4c32-b227-33aee2f66c06 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:51.158250 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926bb9f-f1a0-4c32-b227-33aee2f66c06 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:51.158527 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926bb9f-f1a0-4c32-b227-33aee2f66c06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:51.167325 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926bb9f-f1a0-4c32-b227-33aee2f66c06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:51.168905 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926bb9f-f1a0-4c32-b227-33aee2f66c06 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:51.176894 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1463/2897] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:36:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:36:52.025814 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c4e74-e0a1-4bec-9206-05a97f0084c9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:52.026287 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c4e74-e0a1-4bec-9206-05a97f0084c9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:52.026393 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c4e74-e0a1-4bec-9206-05a97f0084c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:52.044519 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9b1c4e74-e0a1-4bec-9206-05a97f0084c9 None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:52.045696 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9b1c4e74-e0a1-4bec-9206-05a97f0084c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:52.053042 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1435/2898] 10.222.0.67 () {58 vars in 967 bytes} [Fri Jun 24 15:36:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:52.065503 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926bb9f-f1a0-4c32-b227-33aee2f66c06 service swift] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:52.076692 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926bb9f-f1a0-4c32-b227-33aee2f66c06 service swift] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:52.081657 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12daf641-296a-4785-b31c-0a9bfd0cfc2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:52.082295 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12daf641-296a-4785-b31c-0a9bfd0cfc2a service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:52.082580 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12daf641-296a-4785-b31c-0a9bfd0cfc2a service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:52.082916 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12daf641-296a-4785-b31c-0a9bfd0cfc2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:52.091372 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12daf641-296a-4785-b31c-0a9bfd0cfc2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:52.092953 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12daf641-296a-4785-b31c-0a9bfd0cfc2a service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:52.101259 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1464/2899] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:36:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:36:52.638926 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b20fe-8f76-423d-a459-249e0c066f46 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:52.639274 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b20fe-8f76-423d-a459-249e0c066f46 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:52.639582 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b20fe-8f76-423d-a459-249e0c066f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:52.658122 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-df0b20fe-8f76-423d-a459-249e0c066f46 None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:52.659355 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-df0b20fe-8f76-423d-a459-249e0c066f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:52.667355 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1436/2900] 10.222.0.67 () {58 vars in 1009 bytes} [Fri Jun 24 15:36:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:36:52.679145 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12daf641-296a-4785-b31c-0a9bfd0cfc2a service swift] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:52.690504 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12daf641-296a-4785-b31c-0a9bfd0cfc2a service swift] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:52.695629 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350b1f3-c76d-4123-8abd-b397fc88c62d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:52.696316 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350b1f3-c76d-4123-8abd-b397fc88c62d service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:52.696597 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350b1f3-c76d-4123-8abd-b397fc88c62d service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:52.696868 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350b1f3-c76d-4123-8abd-b397fc88c62d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:52.704909 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350b1f3-c76d-4123-8abd-b397fc88c62d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:52.706400 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350b1f3-c76d-4123-8abd-b397fc88c62d service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:52.714467 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1465/2901] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:36:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:36:53.339125 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1045446-4419-4430-9762-8782168f1c06 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:53.339707 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1045446-4419-4430-9762-8782168f1c06 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:53.339707 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1045446-4419-4430-9762-8782168f1c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:53.356671 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c1045446-4419-4430-9762-8782168f1c06 None None] MFA Rules not processed for user `72f00ac4c6a247e485ac52a8dee4f1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:53.357842 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c1045446-4419-4430-9762-8782168f1c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:53.364899 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1437/2902] 10.222.0.67 () {58 vars in 967 bytes} [Fri Jun 24 15:36:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:53.376740 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350b1f3-c76d-4123-8abd-b397fc88c62d service swift] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:53.388129 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350b1f3-c76d-4123-8abd-b397fc88c62d service swift] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:53.393301 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d168c4-e7ac-4edd-8b69-0017976fde2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4732ab3c8c446958092246d16b4b4f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:53.393930 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d168c4-e7ac-4edd-8b69-0017976fde2f service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:53.394208 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d168c4-e7ac-4edd-8b69-0017976fde2f service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:53.394482 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d168c4-e7ac-4edd-8b69-0017976fde2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:53.403281 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d168c4-e7ac-4edd-8b69-0017976fde2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:53.404841 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d168c4-e7ac-4edd-8b69-0017976fde2f service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:53.412631 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1466/2903] 10.222.0.67 () {60 vars in 1384 bytes} [Fri Jun 24 15:36:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:36:55.937658 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1045446-4419-4430-9762-8782168f1c06 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:56.001212 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1045446-4419-4430-9762-8782168f1c06 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:56.005724 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c03ca-0440-4015-87b9-25ed604c74a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:56.006594 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c03ca-0440-4015-87b9-25ed604c74a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:56.006876 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c03ca-0440-4015-87b9-25ed604c74a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:56.007156 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c03ca-0440-4015-87b9-25ed604c74a1 admin admin] PATH_INFO: `/v3/users/212357bccadb4604a7d2855c9be77f75` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:56.008377 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c03ca-0440-4015-87b9-25ed604c74a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=212357bccadb4604a7d2855c9be77f75)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:56.010665 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c03ca-0440-4015-87b9-25ed604c74a1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:56.207448 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:56.209001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-406c03ca-0440-4015-87b9-25ed604c74a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '212357bccadb4604a7d2855c9be77f75'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:56.221636 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-406c03ca-0440-4015-87b9-25ed604c74a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '212357bccadb4604a7d2855c9be77f75'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:56.222314 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-406c03ca-0440-4015-87b9-25ed604c74a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '212357bccadb4604a7d2855c9be77f75'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:56.229470 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-406c03ca-0440-4015-87b9-25ed604c74a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '212357bccadb4604a7d2855c9be77f75'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:56.230102 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-406c03ca-0440-4015-87b9-25ed604c74a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '212357bccadb4604a7d2855c9be77f75'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:56.320068 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1438/2904] 10.222.0.67 () {58 vars in 1223 bytes} [Fri Jun 24 15:36:55 2022] DELETE /identity/v3/users/212357bccadb4604a7d2855c9be77f75 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:36:56.325617 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251467f9-e828-4723-957e-183ff36f599c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:56.326142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251467f9-e828-4723-957e-183ff36f599c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:56.326142 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251467f9-e828-4723-957e-183ff36f599c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:56.343689 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-251467f9-e828-4723-957e-183ff36f599c None None] MFA Rules not processed for user `4f372b5546df46a8836cf73269458aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:56.344859 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-251467f9-e828-4723-957e-183ff36f599c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:56.403967 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1467/2905] 10.222.0.67 () {58 vars in 957 bytes} [Fri Jun 24 15:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:56.417373 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c03ca-0440-4015-87b9-25ed604c74a1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:56.418779 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-406c03ca-0440-4015-87b9-25ed604c74a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:56.477172 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c03ca-0440-4015-87b9-25ed604c74a1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:56.480780 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c57d3-e27b-460a-a950-28668aecb7d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95e7fb8f48dd4348b54e3694a8410bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:56.481431 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c57d3-e27b-460a-a950-28668aecb7d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:56.481713 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c57d3-e27b-460a-a950-28668aecb7d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:56.481992 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c57d3-e27b-460a-a950-28668aecb7d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:56.491001 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c57d3-e27b-460a-a950-28668aecb7d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:56.492453 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c57d3-e27b-460a-a950-28668aecb7d2 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:56.502490 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1439/2906] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:36:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:36:56.769962 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251467f9-e828-4723-957e-183ff36f599c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:56.771415 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-251467f9-e828-4723-957e-183ff36f599c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:56.784030 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251467f9-e828-4723-957e-183ff36f599c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:56.788687 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09864e8a-1fd7-4219-9d43-ce5858f740fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f372b5546df46a8836cf73269458aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e36bcd54fa5438c93e449d16bda5ca3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:56.789555 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09864e8a-1fd7-4219-9d43-ce5858f740fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:56.789838 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09864e8a-1fd7-4219-9d43-ce5858f740fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:56.790122 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09864e8a-1fd7-4219-9d43-ce5858f740fb admin admin] PATH_INFO: `/v3/projects/2c75120127524a4aa9c97a3032c1dd2c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:56.792853 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09864e8a-1fd7-4219-9d43-ce5858f740fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c75120127524a4aa9c97a3032c1dd2c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:56.794912 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09864e8a-1fd7-4219-9d43-ce5858f740fb admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:56.807813 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:56.961402 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-09864e8a-1fd7-4219-9d43-ce5858f740fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c75120127524a4aa9c97a3032c1dd2c', 'request_id': 'req-09864e8a-1fd7-4219-9d43-ce5858f740fb'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:56.967462 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-09864e8a-1fd7-4219-9d43-ce5858f740fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c75120127524a4aa9c97a3032c1dd2c', 'request_id': 'req-09864e8a-1fd7-4219-9d43-ce5858f740fb'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:57.053077 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-09864e8a-1fd7-4219-9d43-ce5858f740fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c75120127524a4aa9c97a3032c1dd2c', 'request_id': 'req-09864e8a-1fd7-4219-9d43-ce5858f740fb'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:57.064225 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-09864e8a-1fd7-4219-9d43-ce5858f740fb admin admin] The token cache is being invalidate because project 2c75120127524a4aa9c97a3032c1dd2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:36:57.068288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-09864e8a-1fd7-4219-9d43-ce5858f740fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:57.068288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-09864e8a-1fd7-4219-9d43-ce5858f740fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:36:57.068288 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1468/2907] 10.222.0.67 () {58 vars in 1232 bytes} [Fri Jun 24 15:36:56 2022] DELETE /identity/v3/projects/2c75120127524a4aa9c97a3032c1dd2c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:37:06.498198 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=39,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:37:07.061067 dv-cinder-835384-11-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:37:49.156077 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c57d3-e27b-460a-a950-28668aecb7d2 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:37:49.157667 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a25c57d3-e27b-460a-a950-28668aecb7d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:37:49.226186 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c57d3-e27b-460a-a950-28668aecb7d2 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:37:49.285228 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a027e-49f6-4ef3-bfa8-9e0b6ade5940 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f612d6ff06fc458e8c1178cbf8321f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06da054e8b7545dfa56780ab56585be8', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:37:49.285905 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a027e-49f6-4ef3-bfa8-9e0b6ade5940 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:37:49.286184 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a027e-49f6-4ef3-bfa8-9e0b6ade5940 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:37:49.286458 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a027e-49f6-4ef3-bfa8-9e0b6ade5940 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:37:49.288225 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ed1a027e-49f6-4ef3-bfa8-9e0b6ade5940 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:37:49.299895 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a027e-49f6-4ef3-bfa8-9e0b6ade5940 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:37:49.301573 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a027e-49f6-4ef3-bfa8-9e0b6ade5940 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:37:49.311399 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1440/2908] 10.222.0.67 () {60 vars in 1365 bytes} [Fri Jun 24 15:37:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:37:59.307041 dv-cinder-835384-11-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=58 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}